Compare rule sets difficulty for human players

Note: there is also a command-line inteface for this tool, scripts/analyze-transcripts-mwh.sh; it has more modes and options than available via the web interface. Documentation is here.

---

Choose one or several experiment plans from the list below. All rule sets included in those plans will be compared.

all_pilot(2 players, 3 episodes) APP/vm/image_test_01(1 players, 4 episodes) Aria01(11 players, 31 episodes) Aria02(2 players, 2 episodes) charles_test(2 players, 21 episodes) cpd/nonrelational_1(3 players, 11 episodes) cpd/relational/relational_1(77 players, 1570 episodes) cpd/relational_1(24 players, 72 episodes) cpd/relational_2(10 players, 35 episodes) cpd/relational_3(6 players, 18 episodes) cpd/relational_order1(3 players, 18 episodes) cpd/relational_order2(1 players, 2 episodes) cpd/relational_order4(1 players, 4 episodes) default(137 players, 984 episodes) ellise(7 players, 12 episodes) ellise_color(3 players, 6 episodes) ep/rule_ambiguity/ambiguity1(41 players, 270 episodes) ep/rule_ambiguity/ambiguity1_ns(106 players, 1718 episodes) ep/rule_ambiguity/ambiguity2(134 players, 1915 episodes) ep/rule_ambiguity/ambiguity3(66 players, 1103 episodes) ep/rule_ambiguity/ambiguity4(483 players, 5993 episodes) ep/rule_ambiguity/ambiguity5(481 players, 1886 episodes) ep/testing/ambiguity4_testing(3 players, 22 episodes) ep/testing/ambiguity5_testing(1 players, 2 episodes) ep/testing/ambiguity5_testing2(1 players, 1 episodes) ep/testing/ambiguity5_testing20(2 players, 6 episodes) ep/testing/ambiguity5_testing_full_20(10 players, 30 episodes) ep/testing/ambiguity5_testing_full_9(2 players, 12 episodes) ep/testing/ambiguity5_testing_full_9_update(9 players, 42 episodes) ep/testing/ambiguity5_testing_full_9_update2(10 players, 59 episodes) ep/testing/count_testing(1 players, 1 episodes) ep/testing/instruction_toggle(15 players, 32 episodes) ep/testing/instruction_toggle_1(1 players, 3 episodes) ep/testing/no_stalemate(7 players, 19 episodes) ep/testing/stalemate(7 players, 17 episodes) gary(5 players, 6 episodes) generalize(6 players, 16 episodes) kevin(2 players, 2 episodes) name_disjunction(203 players, 1760 episodes) name_disjunc_low_first(180 players, 1624 episodes) name_high_first(24 players, 292 episodes) name_low_first(29 players, 364 episodes) name_shape_high_first(21 players, 205 episodes) name_shape_low_first(15 players, 175 episodes) P:APP/vm/image_test_01:APP-short/APP-short-max-feedback(4 players, 5 episodes) P:APP/vm/image_test_01:APP-short/APP-short-no-feedback(2 players, 2 episodes) P:APP/vm/image_test_01:APP-short/APP2-short-some-feedback(1 players, 1 episodes) P:MLC/vm/image_test_02:APP-short/APP-short-max-feedback(2 players, 4 episodes) P:MLC/vm/image_test_02:APP-short/APP-short-no-feedback(4 players, 19 episodes) P:name_disjunction:APP/APP-more-feedback(1 players, 1 episodes) P:pk/position_A:APP-short/APP-short-max-feedback(1 players, 17 episodes) P:pk/position_A:APP-short/APP-short-no-feedback(1 players, 9 episodes) P:pk/position_A:APP-short/APP2-short-some-feedback(1 players, 1 episodes) P:pk/position_B:APP-short/APP-short-max-feedback(2 players, 2 episodes) P:pk/position_B:APP-short/APP-short-no-feedback(1 players, 17 episodes) P:pk/position_B:APP-short/APP2-short-more-feedback(1 players, 1 episodes) P:vmColorTest:APP-short/APP-short-max-feedback(3 players, 8 episodes) pilot01(1 players, 20 episodes) pilot02(24 players, 434 episodes) pilot03(47 players, 852 episodes) pilot04(40 players, 544 episodes) pilot04_1(8 players, 68 episodes) pilot05(33 players, 448 episodes) pilot05_1(13 players, 167 episodes) pilot05_2(1 players, 14 episodes) pilot06(95 players, 933 episodes) pk/colorVshape(7 players, 9 episodes) pk/content_A(25 players, 140 episodes) pk/content_B(28 players, 97 episodes) pk/content_C2(8 players, 45 episodes) pk/content_D(3 players, 3 episodes) pk/dev2024APR19(5 players, 5 episodes) pk/dev2024JAN04(4 players, 10 episodes) pk/dev2024JAN14(12 players, 31 episodes) pk/explore_1(188 players, 2614 episodes) pk/history_A(8 players, 12 episodes) pk/history_B(5 players, 14 episodes) pk/history_C(2 players, 2 episodes) pk/object_C(6 players, 7 episodes) pk/position_A(12 players, 31 episodes) pk/position_A/one_shape_one_color(5 players, 14 episodes) pk/position_B(2 players, 6 episodes) pk/position_C(3 players, 10 episodes) pk/xcolorVshape(2 players, 3 episodes) pk1(10 players, 22 episodes) pkHomeDir(4 players, 11 episodes) pkSingle(2 players, 12 episodes) qt(12 players, 34 episodes) R:APP/combine/comp_A:APP/APP-no-feedback(1 players, 2 episodes) R:APP/combine/comp_AB:APP/APP-no-feedback(2 players, 7 episodes) R:APP/combine/comp_B:APP/APP-no-feedback(1 players, 2 episodes) R:APP/combine/comp_C:APP/APP-no-feedback(1 players, 1 episodes) R:APP/combine/comp_C:APP/APP2-some-feedback(1 players, 1 episodes) R:APP/combine/comp_CD:APP/APP-max-feedback(1 players, 1 episodes) R:APP/combine/comp_CD:APP/APP-no-feedback(3 players, 3 episodes) R:APP/combine/comp_CD:APP/APP2-more-feedback(1 players, 1 episodes) R:APP/combine/comp_CD:APP/APP2-some-feedback(4 players, 8 episodes) R:APP/combine/comp_D:APP/APP-no-feedback(2 players, 6 episodes) R:APP/combine/comp_E:APP/APP-no-feedback(1 players, 1 episodes) R:APP/combine/comp_EF:APP/APP-no-feedback(1 players, 2 episodes) R:APP/combine/comp_F:APP/APP-no-feedback(1 players, 3 episodes) R:APP/cpd/bongards_1:APP/APP-no-feedback(2 players, 6 episodes) R:APP/ep/col_ord_rbyg_1_4:APP/APP-no-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-10:APP/APP-more-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-10:APP/APP-no-feedback(9 players, 32 episodes) R:APP/OCT26/OCT26-10:APP/APP-some-feedback(2 players, 6 episodes) R:APP/OCT26/OCT26-11:APP/APP-more-feedback(3 players, 10 episodes) R:APP/OCT26/OCT26-11:APP/APP-no-feedback(4 players, 13 episodes) R:APP/OCT26/OCT26-11:APP/APP-some-feedback(2 players, 2 episodes) R:APP/OCT26/OCT26-13:APP/APP-max-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-13:APP/APP-more-feedback(2 players, 2 episodes) R:APP/OCT26/OCT26-13:APP/APP-no-feedback(6 players, 6 episodes) R:APP/OCT26/OCT26-13:APP/APP-some-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-1:APP/APP-max-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-1:APP/APP-more-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-1:APP/APP-no-feedback(8 players, 27 episodes) R:APP/OCT26/OCT26-1:APP/APP-some-feedback(3 players, 7 episodes) R:APP/OCT26/OCT26-2:APP/APP-no-feedback(4 players, 12 episodes) R:APP/OCT26/OCT26-2:APP/APP-some-feedback(2 players, 4 episodes) R:APP/OCT26/OCT26-3:APP/APP-no-feedback(5 players, 19 episodes) R:APP/OCT26/OCT26-3:APP/APP-some-feedback(1 players, 2 episodes) R:APP/OCT26/OCT26-4:APP/APP-no-feedback(3 players, 15 episodes) R:APP/OCT26/OCT26-4:APP/APP-some-feedback(2 players, 5 episodes) R:APP/OCT26/OCT26-5:APP/APP-no-feedback(5 players, 14 episodes) R:APP/OCT26/OCT26-5:APP/APP-some-feedback(2 players, 3 episodes) R:APP/OCT26/OCT26-6:APP/APP-max-feedback(1 players, 2 episodes) R:APP/OCT26/OCT26-6:APP/APP-more-feedback(2 players, 2 episodes) R:APP/OCT26/OCT26-6:APP/APP-no-feedback(5 players, 5 episodes) R:APP/OCT26/OCT26-6:APP/APP-some-feedback(3 players, 3 episodes) R:APP/OCT26/OCT26-7:APP/APP-max-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-7:APP/APP-more-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-7:APP/APP-no-feedback(5 players, 5 episodes) R:APP/OCT26/OCT26-7:APP/APP-some-feedback(1 players, 1 episodes) R:APP/OCT26/OCT26-8:APP/APP-more-feedback(1 players, 2 episodes) R:APP/OCT26/OCT26-8:APP/APP-no-feedback(2 players, 6 episodes) R:APP/OCT26/OCT26-8:APP/APP-some-feedback(2 players, 6 episodes) R:APP/OCT26/OCT26-9:APP/APP-more-feedback(1 players, 2 episodes) R:APP/OCT26/OCT26-9:APP/APP-no-feedback(5 players, 5 episodes) R:APP/OCT26/OCT26-9:APP/APP-some-feedback(4 players, 9 episodes) R:APP/OCT26/OCT26-Zach-1:APP/APP-no-feedback(1 players, 1 episodes) R:APP/PAIRS/rule-0:APP/APP-max-feedback(1 players, 2 episodes) R:APP/PAIRS/rule-0:APP/APP-no-feedback(4 players, 15 episodes) R:APP/PAIRS/rule-0:APP/APP2-more-feedback(1 players, 1 episodes) R:APP/PAIRS/rule-0:APP/APP2-some-feedback(1 players, 1 episodes) R:APP/PAIRS/rule-0b:APP/APP-no-feedback(4 players, 4 episodes) R:APP/pk/alternateShape2Bucket_color2Bucket:APP/APP-no-feedback(1 players, 2 episodes) R:APP/pk/colOrd_oneEach_shBckt:APP/APP-no-feedback(1 players, 2 episodes) R:APP/pk/col_oneEach_Ord_nearby:APP/APP-no-feedback(1 players, 1 episodes) R:APP/pk/col_oneEach_Ord_RBYK:APP/APP-no-feedback(1 players, 2 episodes) R:APP/pk/history_A:APP/APP-max-feedback(1 players, 2 episodes) R:APP/pk/history_A:APP/APP-no-feedback(1 players, 1 episodes) R:APP/pk/history_C:APP/APP2-some-feedback(1 players, 1 episodes) R:APP/pk/noRule:APP/APP-no-feedback(1 players, 1 episodes) R:APP/pk/objects_A:APP/APP-no-feedback(1 players, 1 episodes) R:APP/pk/objects_B:APP/APP-max-feedback(1 players, 3 episodes) R:APP/pk/objects_D:APP/APP-no-feedback(1 players, 2 episodes) R:APP/pk/position_Anearby:APP/APP-no-feedback(1 players, 1 episodes) R:APP/pk/position_Aright:APP/APP-max-feedback(5 players, 11 episodes) R:APP/pk/position_Aright:APP/APP-no-feedback(1 players, 2 episodes) R:APP/pk/position_Aright:APP/APP-some-feedback(1 players, 2 episodes) R:APP/vb/1c:APP/APP-no-feedback(3 players, 11 episodes) R:APP/vb/1c_updated:APP/APP-no-feedback(1 players, 1 episodes) R:APP/vb/2a:APP/APP-no-feedback(1 players, 1 episodes) R:APP/vb/2c:APP/APP-no-feedback(3 players, 3 episodes) R:APP/vb/2c_alt:APP/APP-no-feedback(2 players, 2 episodes) R:APP/vb/3a:APP/APP-no-feedback(2 players, 2 episodes) R:APP/vb/3c:APP/APP-no-feedback(2 players, 2 episodes) R:BRM/brm-04:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/1_sm.txt:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/1_sm:APP/APP-no-feedback(3 players, 7 episodes) R:CGS/2_qn:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/3_bltr:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/4_cw.txt:APP/APP-no-feedback(2 players, 2 episodes) R:CGS/4_cw:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/b23-then-b01:APP/APP-no-feedback(2 players, 7 episodes) R:CGS/clockwise:APP/APP-no-feedback(1 players, 2 episodes) R:CGS/color-match:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/_1:APP/APP-no-feedback(26 players, 55 episodes) R:CGS/_2:APP/APP-no-feedback(4 players, 5 episodes) R:CGS/_3:APP/APP-no-feedback(1 players, 1 episodes) R:CGS/_4:APP/APP-no-feedback(3 players, 8 episodes) R:CGS/~1:APP/APP-no-feedback(11 players, 16 episodes) R:CGS/~3:APP/APP-no-feedback(2 players, 2 episodes) R:CGS/~4:APP/APP-no-feedback(2 players, 4 episodes) R:MLC/BMK/farthest_1:APP/APP-no-feedback(2 players, 2 episodes) R:MLC/colOrd_RBYK:APP/APP-max-feedback(1 players, 1 episodes) R:MLC/position_Aright:APP/APP2-some-feedback(1 players, 1 episodes) R:MLC/vm/test-01:APP/APP-max-feedback(1 players, 1 episodes) R:MLC/vm/test-04:APP/APP-max-feedback(1 players, 1 episodes) RU/JF/tht/exp1(21 players, 192 episodes) RU/JF/tht/exp2(1 players, 1 episodes) teaching(15 players, 58 episodes) teaching_6_orders(1 players, 9 episodes) teaching_all_orders(272 players, 3555 episodes) tryPilot04(2 players, 9 episodes) vm//stalemate-01(1 players, 1 episodes) vm/colorVshape(1 players, 1 episodes) vm/composite-01(4 players, 14 episodes) vm/composite-01-super(1 players, 4 episodes) vm/composite-02(2 players, 4 episodes) vm/composite-03(2 players, 5 episodes) vm/image_test_01(5 players, 11 episodes) vm/image_test_02(4 players, 6 episodes) vm/image_test_03(2 players, 5 episodes) vm/jigsaw-01(5 players, 10 episodes) vm/jigsaw-02(1 players, 1 episodes) vm/jigsaw-03(1 players, 2 episodes) vm/jigsaw-04(2 players, 3 episodes) vm/jigsaw-05(1 players, 1 episodes) vm/pilot06_doubling(32 players, 63 episodes) vm/pilot07_doubling(4 players, 14 episodes) vm/Rule_PK1(3 players, 9 episodes) vm/stalemate-01(9 players, 25 episodes) vm20201015(3 players, 17 episodes) vmColorTest(27 players, 78 episodes) vmHomeDir(4 players, 6 episodes) vmPickTest(3 players, 4 episodes)

Stage 1: Transcript processing parameters Learning attainment criterion (enter one): consecutive error-free moves, or R ≥ Default mStar = (You can also enter the value Infinity) Stage 2: data interpretation parameters for M-W Naive: For each rule sets, only include "naive" players (those who played this rule set as their first rule set) Every: Consider each (rule set + preceding set) combination as a separate experience to be ranked. (That is, the R1 data from R1:R2:R3, R2:R1:R3, and R2:R3:R1 are viewed as belonging to three distinct experiences, "R1", "R2:R1", and "R2:R3:R1") Ignore: When viewing a rule set's series, ignore preceding rule sets. (In other words, the R1 data from R1:R2:R3, R2:R1:R3, and R2:R3:R1 are merged, viewed as the same "R1 experience"). Check to use mDagger instead of mStar Compare rule sets (only click once)