?

Log in

No account? Create an account

Previous Entry | Next Entry

Stat Testing - Part I (Cure Formula)

Overview

The December 1.20 patch fundamentally changed essentially every skill and formula in FFXIV - a much needed rework.  With this, it allows players to, for the first time, test for particular formulas without the inevitability that it will just be changed in a few months.  I know that I had personally been looking forward to testing stats prior to the 1.20 patch, but just found it pointless with 1.20 looming.  It has been roughly 1.5 months since 1.20 came out, and the quality of testing regarding stats out on the official forums has been pretty disappointing for the most part.  I hope that this series of posts will help to lay the foundation for future testing on a number of subjects.

For this first post, I wanted to start with the "cure formula" - specifically for Cure and Cura.  I realize that there is already some preliminary testing out there - specifically my own Cure testing in 1.19 and Deli's testing on the official forums.  I want to explain that before we start, I am not going to "build" upon and previously collected data, but intend to start from scratch. 

As a *disclaimer*, I try to involve statistics where they are helpful and necessary, but the overall understanding of statistics would be best described as novice (AP high-school to undergraduate) at best.  My terminology use is especially poor.  Basically, I am in no way shape or form a Robonosto; however, I also don't think that amount of depth and expertise is necessary for this particular set of data.  I will, however, always appropriately note any lack of expertise when tackling data in the future.

As with my other math-heavy posts, I have sectioned off the methodology and discussion sections so that you can simply skip down to the "conclusions" section if the math does not interest you.

_________________________________________________________________________________________
_________________________________________________________________________________________


Methodology

This series of tests involves spamming various tiers of Cure spells on oneself with the help of a simple Cure- /wait- repeat macro.  Gear was used to try to control the stats in question.  Before starting, I made the assumption that the below variables were the only ones that could possible affect the amount cured:

    (1) Level of the Target Cured
    (2) Level of the Caster
    (3) Job/Class of the Caster
    (4) Healing Magic Potency
    (5) MND (Mind)
    (6) VIT (Vitality)
    (7) Magic Critical Hit Rate
    (8) Magic Critical Potency


If one wishes to argue that say STR (or something else not on this list above) affects the amount of HP cured, then that point won't be addressed in this post and would require its own separate testing.  Unless specifically indicated, all tests are performed with an R50 on an R50 (self).  The job/class and stats (Healing Magic Potency, VIT, and MND) will be listed.  It is assumed that unless specified in the raw data that there is no + critical stats with the exception of when THM is used to cure (because THM has an innate trait with "+10 critical").

_________________________________________________________________________________________


Data Collection

Unfortunately, there was not a public parser that fit the needs to auto-collect the data required for this series of tests.  I chose to just manually type in all the values into excel instead.  Below is the summary collection of all tests:

image host

Larger Version


To clarify certain parts of this summary:

    (1) "n" is the number of trials separated into normal and critical hits. 
    (2) The "MIN" and "MAX" were the lowest and highest values seen in the data set. 
    (3) "PREDICT" refers to the average of the minimum and maximum found (more on this later).
    (4) "DEV" refers to the % difference between the maximum and the "PREDICT" average (more on this later).
    (5) "Mean" refers to simply the average of all values in the trial.
    (6) "Crit %" refers to the % of critical hits in that particular trial.
    (7) "BONUS" is the increase in amount cured between the PREDICT of normal and critical as a % of normal.


Please note that not all of these values are relevant or even meaningful for every trial.

_________________________________________________________________________________________


Trial Size Discrepancy

One of the things that should immediately pop up about this data set is that the trial sizes vary widely. 

Assuming the data collection process finds an accurate minimum and maximum, the trial size "n" is only important for precision and accuracy if the true mean is not the same as the average of the true minimum and true maximum.  This concept is absolutely critical for this entire post.  If I cannot convince you this is true, then the vast majority of the data set becomes useless.

Below is a chart of the largest trial in the data set which I short-hand as "Cure_THM50_410H_178V_277M."  I have simply taken every cured value seen in the 294 sample trial (minimum 440, maximum 467) and charted their frequency...

image host

We can clearly see from here that the distribution is hardly "normal" or like a bell curve.  My assumption after seeing this test result is that the distribution of cured values is an even, random distribution across the true minimum and maximum.  This assumption would entail that each possible value has an equal likelihood of occuring; it also entails that the mean and median are the same, but the mode is random.  Important to the testing process, it also means you can find the true mean by finding the average of the true minimum and true maximum.

This is the basis of my assumption that a large trial size "n" is not always necessary so long as the true minimum and maximum are found.  If the true distribution is not even and random as I have assumed, then this would potentially fatally flaw the data set.  However, I feel pretty confident moving forward with this assumption. 

The next question becomes how can we know that our trial MIN and MAX are the true minimum and maximum?  This is where the "DEV" value comes in.  As the trial size "n" increases, the chances of the trial MIN / MAX being the true minimum and maximum should approach 100% (a similar idea to the "Law of Large Numbers").  After conducting a number of the trials, I found that at large 100+ sample sizes, the DEV value approaches around 3.00%.  This is only my observation, but is the basis for why I will immediately cut off a trial if the DEV > 3.00% even at small trial sizes.

During each trial, I had to decide what would be my "cut-off" scenario where the trial would end.  For the vast majority of players who attempt to conduct similar testing, this comes down to a predetermined trial size of a "large number" in order to get a more precise mean.  For these tests, I chose to use my cut-off algorithm as:

    (1) A trial can be immediately cut off or completed if the "DEV" value is greater or equal to 3.00%
    (2) If a trial's "DEV" value is less than 2.91%, at least 100 trials should be conducted to ensure precision.
    (3) The "mean" or trial average is provided for every set of trials, but is not used in analysis.


At this point, I'll start talking about actual data analysis.  But for the sake of discussion and openness, I want to stress that the above assumptions were what I based my data collection process on; if they are false or questioned, it potentially taints the results below.

_________________________________________________________________________________________


Effect of MND on Non-CNJ Classes

I'll start the discussion with a test where all variables were evened out as best as possible with the exception of the class (between CNJ and non-CNJ classes).  In the data set below, I chose THM because is was the only reasonable job that could match Healing Magic Potency given the gear I had available.  Unfortunately the MND stat is not well controlled, but we take a look at the MIN / MAX / PREDICT values, we find that even though the THM had higher MND, the CNJ still cures for a significantly higher amount.


image host

The reasoning behind this has to do with the fact that MND itself has absolutely no influence on the amount of HP cured when using Cure on non-CNJ classes; although the Healing Magic Potency gained from MND still has an effect.   Admittedly, I chose to perform this test first because I had strong reason to believe this to be true based on other testing (INT etc.).  I think this observation appears pretty hard to miss at first; however, may have been difficult to elucidate due to the fact that increasing MND still increases Healing Magic Potency, which DOES still have an effect on non-CNJ classes.

I will be the first to admit that 1 trial (especially one where the variables are not even truly even) is not ultimately sufficient to prove this concept.  However, I hope that this in combination with future testing on INT will show that certain modifiers involved in formulas are exempt when using non-main classes.

_________________________________________________________________________________________


Effect of Vitality of Cure

Very simple test run on non-CNJ classes where all stats remained the same other than a large VIT increase.  This required a changing of classes (THM and GLA); however, I think most people will be fairly satisfied with this lapse as they are still non-CNJ classes.  Data set below:

image host

We find that there is a +9.5 damage increase for a +84 increase in VIT.  Note that the DEV may indicate that this value may not be that precise (especially given the low sample size corresponding to this).  I personally used this test to show that I needed to control for VIT or the values would be altered and that was about it.  I feel like it is important to know that VIT is a modifier, but it seems unlikely that a "VIT build" would be useful for actual healing builds given the incredibly modest increases (0.1-0.15 ratio). 


_________________________________________________________________________________________


Effect of Healing Magic Potency on Cure and Cura

This next set of comparisons involves increasing Healing Magic Potency while maintaining static VIT and MND.  As we are all aware, increasing the MND stat by 4 also increases Healing Magic Potency by 1, so this was slightly difficult to control without a set of gear specifically made for this test.  I worked out a set with the help of the LS that gave +41 Potency with static MND and used this to test for both Cure and Cura.  In addition to this, I also ran a small test involving Cure on a Lv2 character (both on CNJ and non-CNJ classes).  Data sets below.

image host

Based on these data sets, we find that:

    (1) Increasing potency from 424 to 465 gave +51.5 increase in average to Cure (1.25 HP per 1 potency)
    (2) Increasing potency from 424 to 465 gave +102.5 increase in average to Cura (2.50 HP per 1 potency)
    (3) The 1.25 HP to potency ratio for Cure is not static based on Lv2 cure data.


If we simply take the average per potency increase for the first set of data on Cure and Cura, we get really nice numbers as 1.25 and 2.50.  However, just looking at this gain logically, if you had 424 potency and the increase was static at 1.25 HP cured each, you would end up with at least 424 x 1.25 = 530 (which is nearly the exact average without adding in VIT and MND modifiers).  Running the calculation on Cura gives you 424 x 2.50 = 1060 (again near the average prediction).

This leads to the interesting question of is the ratio (1.25 or 2.5) of HP cured to potency influenced by the VIT and MND modifiers already?  Or are they separate and the discrepancy shown in the Lv2 cure data based on the potential idea that potency itself is tiered (meaning adding +41 from 1 to 42 is different from going from 424 to 465)?   Bringing in another data set looking at potency on non-CNJ jobs:

image host

In this data series, I increased the potency by increments of 10/20 while controlling VIT, but no necessarily MND since it's shown earlier that it has no effect on the formula when looking at non-CNJ classes.  We find that the ratio of increase in these tests is closer to 1.1 (instead of the before-mentioned 1.25); however, the increase in potency as a % seems to follow the increase in amount cured pretty precisely.  For instance, going from 420 to 440 is a 440 / 420 ~ 4.76% increase.  The amount cured went from 451 to 472, or a 472 / 451 ~ 4.66%.

Based on this data, I think it would be pretty safe to say that increasing potency with fixed MND (if CNJ) and VIT stats will grant a straight percentage increase.  This is probably a more accurate description of its effect over say a "ratio increase".  This is also a pretty significant distinction when eventually going after a formula portrayal of the "cure formula."

_________________________________________________________________________________________


Effect of Mind on Cure and Cura

The last basic stat increase we have to talk about is mind.  As mentioned in the previous section, it appears that potency gives a general percentage increase to the amount of HP cured, meaning MND (when used on CNJ) will directly affect its "ratio of increase" so to speak.  Let's take a look at the data for this:

image host

For a +62 increase in mind, we have a +15 increase in Cure (~0.25 ratio) and +31.5 increase in Cura (~0.50 ratio).  To quickly see if there was any sort of "tier breaking" going on, I ran another quick test with a +8 increase in mind on Cura and got a +5.5 increase in Cura, which roughly corresponds to the % increase seen in the +62 test.  In reality, I think the ratio increase here will change and increase as potency increases, but for the values seen here, a 0.25 ratio for Cure and 0.50 ratio for Cura are decent approximations for potency values in the 400s.

Again, I feel that there is a level of complexity to the actual formula here that makes saying "0.25 MND to HP cured ratio" an oversimplification, but for the stat values seen at R50, I think it's a fairly accurate statement in this case. 

Lastly before moving on, if there is a "tier" system to mind increases, there is no indication of one for the values tested here (260s to 320s).  This does not mean one does not exist, but there is just no evidence in the limited test range seen here.


_________________________________________________________________________________________


Effect of Magic Critical Potency / Magic Critical Rate

If we just take a quick look at the "BONUS" column on the overall data collection earlier, we find that most of the critical bonuses fall in the range of +22-23%.  This poses the question of how much critical potency and Sagacious Might increase this bonus.  I set out to run only a quick rudimentary test to get an assessment of this bonus increase, but did not feel ready to tackle anything near the depth of the "normal" cure formula.

image host

We find here that the "BONUS" increases from about 123.70% to 133.85%, which comes out to a rough 8.2% increase in the total amount of HP cured on criticals with +58 potency.  I will say that this test may be "tainted" by the fact I performed it on THM though.  If we look at the critical trait on THM, it says on the English client "Enhanced Magic Crit Potency - Increases critical rate of spells by 10."  I found this a bit confusing because it appears to be talking about potency (aka "BONUS") in the trait name, but the description sounds like it's talking about rate (aka crit %).  Is it one, the other, or both?  Because I don't know this, it's hard to truly assess the increase of +58 potency here.

We can however at least take a closer look at the critical hit rate.  If we take all the trials conducted in the entire data set and pool the critical rate, separating out those conducted on THM (with the +10 rate trait) and those not on THM (with no critical bonus), we get a nice looking table like this:

image host
THM rate = 10.80%  |  Other rate = 7.71%


From this 2 by 2 table, we can run to "Chi Squared" Test to see if there is actually a statistically significant difference in the critical rate for THM due to trait versus other jobs without the +10 trait.  When we run this, we get a Chi Squared of 4.3968 and a p-value of 0.036, which is less than alpha=0.05.  All this really shows is that the Critical Rate trait does something, but it can't tell us exactly how much it does.

This concept of critical hit bonus and rate will be explored as we look at other formulas like the MATK/INT formula.  There is a level of complexity involving dLVL that I just don't want to touch this early on in the stat testing until we have enough built information.




_________________________________________________________________________________________
_________________________________________________________________________________________


Conclusions

I will try to summarize the "important" game-application findings found from the discussion of the cure data set; however, be aware that I will be extremely conservative when making calls here since most of the results are at best estimations of what's really going on with the formula.  That being said, I feel like there are still some good things to know even without the true formula.

    (1) Mind itself does not affect the HP gain from Cure when used on classes other than CNJ.

This still means that if you increase the Mind stat on say THM, it will increase your Cures, but it only does this because you also get 1 point of Healing Potency for every 4 Mind you add.  However, you will only get additional bonuses to cures for the mind stat itself if you are on Conjurer.  As a preview to future stat posts, I can go ahead and safely say that this "modifier loss" also applies to other formulas such as Second Wind and attack magic.  For example, if you use Second Wind on PGL, INT is a modifier, but is not for non-PGL classes.  INT is also a modifier for Thunder on THM, but other classes do not get this INT modifier if cross-classing Thunder, etc.

    (2) Adding +1 Healing Magic Potency adds roughly 1.25 HP on Cure and 2.50 HP on Cura when on CNJ. 
          It adds roughly 1.10 HP per potency when on non-CNJ classes.  Be aware this is an oversimpliciation.


I chose to write this in terms of "ratio of increase" despite the fact I kept calling it an oversimplification in the body of the post.  I feel like for the average player, ratios make more sense when it comes to gear choices.  I feel that in the future, a formula estimation will be inevitable and at that point, the true relationship will replace the "ratio explanation."  From the mathematical point of view, it looks like potency is a percentage increase to HP cured, meaning if you increase your potency by 20% keeping everything else the same, your HP cured will go up by 20% as well.

    (3) Vitality is a minor modifier to the Cure formula.

Vitality appears to add roughly 1 point to Cure for every 8-10 points added to it.  It's probably too small to make a difference or make it worth having a "VIT build," but is good to know for the purposes of controlled testing.  I did not test the effects of VIT on Cura, but made the assumption that it would not make a big difference relative to potency and Mind.  It's important to just know this modifier exists.

    (4) For CNJ only, +1 Mind adds roughly +0.25 to HP to Cure and 0.50 to Cura.  Much like my explanation
         for Healing Magic Potency, this is an oversimplification of what's really going on.


To stress, remember that this only applies if you are on the CNJ class.  Other classes do not benefit from a mind modifier on cures.  This "ratio explanation" is an oversimplified version of what's really going on and the ratio will increase as your Healing Magic Potency increases.  When making gear choices, also remember that adding 4 mind grants you 1 potency.  This 0.25 / 0.50 ratio increase described above does not take that into account.  When you do take that into account, the ratios are actually more like 0.5625 for Cure and 1.125 for Cura.  You could summarize stat gain ratios as:

* Cure on CNJ
    1 potency = 1.25
    1 mnd = 0.5625

* Cura on CNJ
    1 potency = 2.50
    1 mnd = 1.125

* Cure on non-CNJ
    1 potency = 1.10
    1 mnd = 0.275


    (5) When the caster and target are both R50 with no critical bonuses, the % bonus on critical cures is roughly
         an increase of 22-23%.  The rate is roughly 7.8%.  Limited testing on critical potency bonus so far.


The amount of testing and discussion for critical was fairly limited in this post because it's a bit more complex than I'm ready to get into at this point.  The best I can offer for this post is that that THM trait that gives +10 critical rate does work for cures and that if you add +58 critical potency, the critical bonus increase jumps from about 22-23% to 34-35%.

Whew!  That's about it for this.  Just a couple notes to end things.  Critique is always welcome; the harsher the better.  The testing and discussion shown here is limited to the point where we can't derive a usable "cure formula" as of yet, but I am confident that with some community help or just pure time to test more, one will be found fairly soon.  Also I'm sorry I didn't get to test Curaga. 

Lastly, for those that don't play XIV but still follow me and want my advice about the game: wait for 2.0 in Q4 2012.  The game is fun right now but that's only because I just played 1.5 years in the train-wreck that was this current game's predecessor.  If you play now, it's paid beta-test.  If you still want to play though, I'm on Figaro!

Til next time.


_________________________________________________________________________________________
_________________________________________________________________________________________

Comments

( 7 comments — Leave a comment )
(Anonymous)
Jan. 29th, 2012 01:27 pm (UTC)
Questions
Ok tell me if i'm wrong but from your post Healing pot clearly own MND for cures on CNJ right?

Also you said that 20% healing pot = 20% HP restored

Did you test if there was a cap like in FFXI the cap was 50%

Thanks and good job as always
kanican
Jan. 29th, 2012 05:32 pm (UTC)
Re: Questions
If there's a cap for potency I was unable to find one. I know people with 500+ potency and it doesn't seem like the return is any less from eyeballing.

From this testing, yes, potency is clearly the stat of choice over MND when both are available. Using the "ratios" it's basically:

Cure on CNJ
1 potency = 1.25
1 mnd = 0.5625

Cura on CNJ
1 potency = 2.50
1 mnd = 1.125

Cure on non-CNJ
1 potency = 1.10
1 mnd = 0.275
(Anonymous)
Jan. 31st, 2012 12:52 am (UTC)
from what i saw with deli's info, there were tiers. basically we need someone with like +60-80 healing pot to check if there are diminishing returns to that point or if u hit another tier.
kanican
Jan. 31st, 2012 04:14 am (UTC)
Can you give me a link to Deli's post? I'm having trouble finding it, but I don't recall there being a tier discussion that led to this conclusion?
delicatestar
Feb. 7th, 2012 03:44 am (UTC)
Does water have any effect on cure (a.k.a Wand Of Tides)?
kanican
Feb. 7th, 2012 06:22 am (UTC)
No, Deli tried to show this in his testing on the official forums. But if you want to see for yourself, you can check YG, which .dat mines info and you'll find that cure is neutral element (not water).
(Anonymous)
Feb. 19th, 2012 01:02 am (UTC)
im back about deli's stuff. http://forum.square-enix.com/ffxiv/threads/33293-1.20-Cure-Test-Mnd-vs-Water-Elemental-Pot-vs-Healing-M.-Pot there it is, on the first page, im not sure how many tests he did but there seem to be dips within his avgs changing the stats around. is it possible that there is a "next" tier to hit, or more realistically a tier that costs too much to hit so there is some sort of optimal place to hit and then focus on crit or hp for more mp back on that hp to mp move?
( 7 comments — Leave a comment )