Rewording and Reworking 8Values From The Ground Up

[JoshuaKimsey]

So I have been thinking of some ways to help improve the quality of the test and solve some commonly seen complains about it.

Firstly is the perceived bias many say they see in the test. Most individuals I've heard criticism from complain of a left-leaning bias in the test. I struggled to find a way to easily quantify this supposed bias, until I found a rather simple, yet elegant solution. Clicking the same answer on each question, for example "Strongly Agree", the final results should roughly even out to be 50/50 in each category, or lose to it. This is because in a properly balanced test, a survey topic should be approached by asking two questions. Each question is the opposite of the other, meaning that if answered the same way, the results would cancel out to a net-zero change in the final results for the survey. This is an easy way to tell what bias a survey might have, and how badly it is skewed in one direction or another. As an example, I used this method on the Political Compass test and you can see the results below:

As you can see, the economic axis of the graph came out to be exactly 0, but the authoritarian/libertarian axis has an obvious, and rather large, skew towards the libertarian end of the compass.

Following this procedure, I used this method on the 8Values test, selecting "Strongly agree" too all questions, and you can see the results below:

You can see that the diplomatic axis is decently balanced, at only a little over 1 percentage point difference. The societal axis, though a few percentage points, is within what I would consider to be an acceptable margin or error. The Economic and Civil axises though are far beyond what a margin of error should cause. It's obvious that there is an imbalance of questions for those two parts that leads to a very large, 20 point gap total between the two sides of each axis. (Please note that selecting all "Strong Disagree" to the questions will result in a mirror opposite of the results shown above, with the same numbers being reversed on the axis scales.)

So firstly, we need to go back and we work the questions to include two for each basic idea being asked, each having an opposite view from the other. For example, one question could be, "I support single-payer, universal healthcare" while the second one could be "I do not support government welfare for medical needs". This suits the opposing view for the same topic, and also insure that people who fall below the "Universal healthcare", but above "No Welfare" have an option to click. This also means that there will always need to be an even number of questions in the test, though I don't see why that should be an issue.

Secondly, we need to get rid of questions that contain double negatives. Double negatives might be ok in grammar, but in a survey they will lead to confusion about what option to pick. For example, instead of saying, "No authority should be left unquestioned", say, "All authority should be questioned", as was mentioned in another issue thread #64

Finally, I think one other thing that would go a long way towards helping to ease fears of bias is to have the questions be randomized each time a user takes the test. This shouldn't be too terribly hard to achieve with just some JavaScript, an Array, and a random number generator. This would help make the test less predictable and more unique each time someone takes it.

These are just a few of my suggestions to help make the 8Values test truly great! Let me know what y'all think of these suggestions, or if you have any additions to add to them! 😃

[JoshuaKimsey]

Definitely look for some input from the other Contributors on this topic! @TristanBomb, @elifoster, @gervasiocaj, and any others wanna jump in and offer some input on this idea? 🙂

[TristanBomb]

I definitely agree with most of the changes you suggest, but I also dispute that "what happens when you agree to everything" is a sure-fire way to test the bias of a test. A more insidious issue is that questions might very well be worded in a way that lends users to respond in a certain way; I think one way to resolve this is to just have multiple users writing the questions. As the only one to write questions (so far), I've tried to avoid letting my views bias the test as much as I can, but I can't guarantee that I was successful. For reference, my views are consistently left-wing. Randomizing the order of questions probably won't affect the bias of the test, but it might still be a good idea.

I'd also like to comment on the ways I think the test might be biased. This isn't based on hard data or anything, but just what I've observed. I think the economic axis definitely reports values left of the "true value," but I actually think the authority axis is fine. Yes, clicking agree to everything resorts in a more authoritarian result, but most people are usually inclined to click the more liberty-oriented responses, so I reckon it balances out. The diplomatic axis, as you state, is fine; I'm not really sure about the society axis.

Finally, while we're discussing complete reworks, I/we/someone really needs to draft a better ideology matching system, because frankly, the current one just doesn't work very well. I have some ideas, but I'll need more time to develop them.

[JoshuaKimsey]

I will admit, that isn't an aspect I thought about. However, you are entirely correct that even though the individual axis's are evened out, they could still be skewed one way or another. However, this might be hard to fully find as it may not be immediately obvious from the outside. Even still, just doing what I did can show more obvious preexisting bias that is in the test.

With the randomization, this would help cut any preexisting bias an individual might have coming into the test if they found the test through someone else, like Sargon or Esoteric Entity. This will help people to read the questions and answer for themselves, instead of falling back on what someone else might have done.

I do agree as well that the ideology matching system does need some help. For one, it's far to exact! For example, my results show Libertarian Capitalist, but I can say honestly that the Civil Axis is exactly one number away from making me an Ultra-Capitalist. If I answered one question even just slightly different, I would get an entirely different result. What might be an optimal substitute to this exact system would be maybe a list or graph showing what the top three most related ideologies are that someone matches. So for me It'd probably Libertarian Capitalist, Ultra Capitalist, and maybe something like Anarcho-Capitalist, or whatever the next best alternative is for my axis matches.

Another issue is presented though, I'm actually not a Libertarian, but instead I'm an Objectivist. While technically Libertarian Capitalist is decently close to Objectivism, they are not the same. For me, the most important graphs would be the Market and Civil axis. The Societal axis is less important, but I'd imagine it'd trend towards being progressive, as you can see in the image above. The Diplomatic axis could prove troublesome though. Many Objectivists might place themselves in the "peaceful" category. However, when that's on a scale that includes Internationalism at the far end, that idea sorta falls apart. So I don't honestly know how to handle the Diplomatic axis when trying to determine something like Objectivist, or even Libertarian Capitalism for that matter.

These are things that need to be sorted out in due time. How to sort out some of these issues is beyond my scope of reasoning at this point. However, I think some things can be fixed decently easy! Making the test less biased, or at least seem less biased, would go a long ways towards fixing some of the major griefs with it I heard! As for the ideology matching, is there any kind of list containing all of the major political ideologies you could use as a blueprint to make the list/graph style of ideology matching for people? That might save you a lot of work when remaking that!

[Berkmann18]

Any update on that?
If help is needed, I wouldn't mind.

[brmbrmcar]

I created a pull request to fix it slightly ages ago.

[Berkmann18]

It's not referenced it seems.

[brmbrmcar]

#70

[Berkmann18]

It's still open tho :/.

[brmbrmcar]

I know because nobody has bothered to look into it in-depth/give a full opinion.