Differences

This shows you the differences between two versions of the page.

--- research:rater_instructions [2024/02/11 19:54] – [How to count boundary clipped objects] admin
+++ research:rater_instructions [2024/02/11 20:08] (current) – [Different sizes] admin
@@ Line 42: / Line 42: @@
 The AI model was asked to draw images with 1 through 16 of the different objects. However, the AI model often gets the number wrong, and we'd like to know exactly how it goes wrong. Therefore, our job is to count all of the objects in the image, even if there are more than 16.
-However, sometimes the AI model's image is so obviously wrong that it just doesn't seem worth it to count all its mistakes. So, **if and only if we're sure** that there are more than 20 objects in an image, we can click an option for "Definitely more than 20" which will make the response slider-thingies disappear for that trial, so that we don't have to give a precise answer.
+[{{ :research:survey_phone_screen_top.png?300|An image to rate and the "more than 20" question}}]
-{{ :research:survey_phone_screen_top.png?200|An image to rate and the "more than 20" question}}
+However, sometimes the AI model's image is so obviously wrong that it just doesn't seem worth it to count all its mistakes. So, **if and only if we're sure** that there are more than 20 objects in an image, we can click an option for "Definitely more than 20" which will make the response slider-thingies disappear for that trial, so that we don't have to give a precise answer.
 Recommendation: If it's not immediately clear whether there are more than 20 objects, start to count them, and then stop if you get close to 20 and there are still lots of objects left.
@@ Line 80: / Line 80: @@
 The second response slider-thingy asks for the number of boundary clipped items. "Boundary" just means "the edge of the picture" in this case. If less than **50% of an object** is shown in the frame of the picture, it counts as boundary clipped.
-{{:research:boundary_clipping_example.png |}}
+[{{ :research:boundary_clipping_example.png|Boundary clipping examples}}]
 Note: Objects that are partly occluded by (hidden by) //other objects// **don't count** as boundary clipped.
+In the example below, I counted 13 medicine balls, with 2 boundary clipped.
 ====== Consistency of objects ======
@@ Line 94: / Line 94: @@
 Imagine that there is a picture of two hot air balloons that are the same real world size (say, 60 feet tall). If one is far away, it might only take up 1/4" on the screen, while one that is near the front of the picture might take up 2". **These two balloons would be rated to be //the same// size.**
+[{{ :research:survey_phone_screen_sliders.png?300|The slider-thingies and some of the check boxes}}]
 Recommendation: Allow a little tolerance so that objects that are very nearly, but not exactly the same size can count as "the same size." When objects are noticeably different size, then rate them as "different sizes."