Skills Testing

Testing Plan

Back to Skills

User Testing Interviews of Skills and Skill Tree game experience

A qualitative evaluation approach of using open ended interviews when interacting with the mechanics was taken. This involved in order:

• Allowing the player to become overall familiar with how the game plays without skills after a short explanation of player goals and game purpose
• Introducing the player to the skill mechanic through interview prompts

The following prompts were given during testing from the main game screen:

• Please find and open the skill tree
• Please equip one of your ultimate skills and 2 of any other skill that interest you
• The controls for the skills are j, k and l, please feel free to use all skills however you would like for a minute and then I'll ask you some questions about your experience

We conducted 5 interviews of the following questions:

• 1: Do you think the skills you used were useful? Why?
• 2: Do you think the skills you used were fun to use? Why?
• 3: Did you find it easy for you to equip the skills you wanted? Why?
• 4: Did you think that it was easy to find where to equip skills?
• 5: Did you find equipping skills easy?
• 6: Do you have any improvements you might suggest for improving the skill tree?

Most of the responses we received were positive responses. Below are the notable responses that we used to justify the additions to the game this sprint (numbered based on the interview questions above):

• 1: All of the skills felt useful once I understood what they were, but I couldn't really understand what the skills were until I had them equipped and tried them. Follow Up Question from interviewer "What do you think should be improved to make this easier to understand?" Response Although I like discovering new skills, they're complicated enough that I can't really understand them just from the icons, you need like a wiki or something.
• 1: Some of them seemed more useful than others, I couldn't really tell when some of them were working.
• 2: All responses to question 2 were positive (Note, potentially too much of a leading question and should rephrase to not be positively toned and mention fun)
• 3: Equipping the skills was really easy, but I just didn't know which ones to equip from the skill tree.
• 4: Kindof, the skill tree thing in the top right was the only thing that seemed clickable on the screen so it made sense to click there. Follow up question "If you had other options do you think you would have chosen the same option you picked this time? and would you suggest some improvements?" I'm not sure honestly, but I'm not sure what it was supposed to be at the moment so probably not.
• 5: Everyone answered positively
• 6: I feel like the levels don't make sense to me since there's no levels on the screen and I can't see the level on the skill tree, maybe you should get rid of the level text on the skill tree or improve it to make more sense.
• 6: The skill tree icons need clarifying I think, and the levelling system feels unclear yeah.

Changes to game as a result of above user testing:

From these user responses, there were a number of actionable changes to the game desgin. We extracted this to be:

• Improve clarity of skill tree icon from main game screen. This was listed as not looking like a book in the above responses, and sometimes was difficult for players to find. This was resplaced with a 3d book in the resultant changes.
• Improve the clarity of skill activation, at the time of testing some of the skills were without relevant animations. Such as the bleed and the root skills, and some of the animations were the same as others (such as the charge attack and the dash), these animations are a vital part of the responsive feeling of player interaction with the skill system, and so consistenly making sure that there differing animations for all skills will improve player understanding and enjoyment. These animations were added in the resultant changes to aid in user understanding and increase the overall investment of the player.
• Improve the clarity of skill icons, and or add some text description of each skill icon to the skill tree, so that users can better understand which skills they are equipping from the skill tree. This was probably the most important improvement and the most notable from the above interview questions and so this was prioritised as opposed to other functionality. The solution to this was to add a hover tooltip to the skill tree for each skill icon, so that the player could read and understand the skill from the skill tree without having to exit the menu and test the skills to understand how it works. Improvement to the icons was not as much as a priority as we identified text as being the most productive way of aiding player understanding and the most efficient way of conveying the skill information.

Future Testing Plan

The approach during this sprint was to fix the most important user problems identified in user testing interviews above. We have made large improvements based on the above interviews but have yet to be able to validate that the improvements entirely solve the problem identified. In future, as relating to the improved functionality, the following should be tested:

• The player can identify the functionality of skills from the skill tree
• The player understands the skills entirely
• The current icons portray the best version of the skills. This is likely one of the most important aspects of validation that wasn't touched on during this sprint, and as a result visual representations of the skill icons should be thoroughly tested in future.
• The player understands which skills are equipped to which skill slot and how to activate each skill. (This was attempted to be tested in the interviews conducted, however the interviewees didn't identify this as a problem, and potentially further interviews should put an emphasis on evaluating this aspect of the skill system)
• The skill animations best portray the skill being used, and are the most engaging visual implementations of this. (This should be tested against other designed animations, where designing multiple animations for each skill was infeasible within the alloted time for sprint 4).
• The experience system is appropriately scaled for the difficulty of enemies. (This should be encorporated with enemy user testing).
• All skill functionality should be thoroughly tested for bugs, especially considering the potential for each skill's interaction with the other. There is a very large number of permutations of possible equipped skills at once and so this was too difficult to bug test visually and mechanically in the alloted time.

PlayerSkillsComponent class

Code Testing

This code is heavily tested in our testing plan. This is tested by expected values of movement changes via movement altering skills, giving expected results of vector manipulations. Testing the internal variables and the changes between skill states is also an important consideration to this testing plan. This testing plan aims to always test all functionality within the PlayerSkillsComponent class, as this is where the functionality of the player skills is testable and mostly decoupled from the player actions class for effective black box testing. Other classes designed such as playerSkillAnimation controller is aimed to be tested through visual inspections of animation digraph traversal.

Visual Testing

Some skills are unable to be thoroughly tested for player experience through only code/unit tests. As such these visual inspection on the functionality of skills and their skill states in relation to the rest of the game is required. This can include visual tests such as making sure the player can move during a dash, or making sure that the skill ends/performs the right function within the game environment. This will change for each skill design, and will be more strict as the possible combination of skills is increased, as this will make the skill class more likely to break functionality with multiple complex vector operations and player attacks. These will be added to the visual checklist for next sprint, but is dependent on the type of skills set to the different skill slots. Visual Tests For Skills:

• Do all animations end and disappear after completion without haning
• Does the skill tree assign all skills
• Does the skill tree clear button clear all skills
• Does the skill tree lock skills above the first row when the player hasn't killed any enemies
• Does the skill tree open with the equipped skills after equipping skills and closing them
• Does the shield animation start and stop within one animation cycle
• Does the teleport animation trigger during charge-up of teleport
• Does the dodge allow players to move quickly only in the sideways directions (relative to initial dodge movement direction)
• Does taking damage during the dodge increase the movement speed of the player for a short time
• Does the dodge interrupt the player dash
• Does the projectile spray fire on keypress and does each projectile have a different rotation corresponding to its movement direction.

PlayerSkillAnimationController class

Because this class requires a full set up working environment the testing plan for this requires that users visually test the code. This animation checklist is included in the list above.