ICILS is an international assessment of computer and information literacy taken by 8th grade students. It consists of two assessments, Computer and Information Literacy (CIL) and Computational Thinking (CT) which is an optional test. CIL measures how well students could use digital devices such as computers to collect, manage, create, and exchange information digitally1 and CT measures how well students could “develop algorithmic solutions” to real world problems that could be carried out by a computer.2
The International Association for the Evaluation of Educational Achievement (IEA) develops and oversees administration of ICILS. ICILS has been conducted in five-year intervals, starting in 2013. The U.S. participated in both the CIL and CT components of ICILS in 2018 and 2023.
IEA is currently preparing for ICILS 2028. Learn more about ICILS at https://www.iea.nl/studies/iea/icils.
Solve a CT test question
Want to understand the kinds of tech challenges children are learning? Try the Farm Drone Computational Thinking item and find out.
Farm Drone task
The Farm Drone task represents a visual coding Computational Thinking (CT) task. It was developed for ICILS 2018, described in the 2018 international report as an exemplar CT task, and then released after 2023.3 This task involves dragging and dropping code blocks into (or out of) a large gray workspace. The workspace contains a series of code blocks, and the student’s job was to debug the code. Each code block was designed to perform a specified function affecting the output of the task.
At the IEA ICILS 2023 assessment page, click on the Farm Drone button.
You will be taken to a login page (see screen shot below) for this demonstration task. Type in ‘Farm Drone’ and ‘ICILS.’ Good luck!
At the start of Farm Drone, students watch a tutorial and read some instructions, including information hinting that efficient use of code blocks is a goal to strive for.
The interface might not look as complex as today’s most popular games, but it does bear a resemblance to The Farmer was Replaced, a game where users program in Python to advance through a game that is set on a virtual farm.4 IEA built the interface for ICILS to mirror computer applications students were likely to encounter in 2018.5
In the actual 2018 assessment environment, a task like Farm Drone would have been one of two CT modules presented to students. They would have 50 minutes to complete the two CT modules and would have taken the CT test after completing the CIL test and a student questionnaire. Farm Drone comprised nine subtasks, ranging in complexity from low to high.
Interpret results from the Farm Drone task
How student responses to Farm Drone were scored
Students could receive a score of zero, one, two, or three points on this task.
- Students who could complete all of the objectives but with many more code blocks than the minimum necessary were awarded one score point.
- Students who could complete all of the task objectives with only a few code blocks more than the minimum necessary, by using the repeat and if statements together, were awarded a score of two.
- Students who were able to correct the algorithm using the minimum number of code blocks were awarded a score of three. These students demonstrated clear control over the non-linear conditional logic of the algorithm.
Interpreting the results on the CT scale
Because 2018 was the first year CT was assessed and the number of items was limited, CT items were ranked by difficulty and separated into regions with an equal number of items in each group.6 The 2018 CT scale was divided into three regions: lower (below 459 scale points), middle (between 459 and 589 scale points inclusive), and upper (above 589 scale points). On the ICILS scale, higher score points represent increasing task difficulty, so the number of points a student earns indicates the level of complexity they can handle. Easier tasks that involve basic, guided digital actions correspond to the lower score region while more demanding tasks requiring critical thinking and advanced information processing correspond to the higher score region on the ICILS scale. Scoring three points for the Farm Drone task would put a student in the upper region, above 589, for the task. If they scored one or two points, it would put them in the lower region.
Figure 1. International Computer and Information Literacy 2018 Computational Thinking scale score range.7
Table 1 below shows the score points that could be achieved, the level of Computation Thinking (CT) scale difficulty the task represents, the CT scale region to which the score was assigned, and the percentage of students taking the 2018 ICILS CT assessment who received one, two, or three out of three points for responding correctly to this task. The task was designed to demonstrate students’ ability to operationalize solutions and develop algorithms, programs, and interfaces. Please refer to the ICILS 2018 framework for more details on the framework and the ICILS 2023 international report for a thorough breakdown of the Farm Drone module subtasks.8 9
Table 1. Example CT Task 1 with framework references and overall percent correct across participating education systems, 2018
| Score | CT scale region | CT scale difficulty | ICILS 2018 average percentage correct responses |
| At least one of three points | Lower | 353 | 86 (0.3) |
| At least two of three points | Lower | 396 | 77 (0.4) |
| Three points | Upper | 613 | 27 (0.5) |
| ICILS assessment framework reference | |||
| 2.2 | Operationalizing solutions | ||
| Developing algorithms, programs, and interfaces | |||
Reproduction of Figure 4.1. from Preparing for Life in a Digital World international report (2019), p. 117
Table 2 shows, by education system, the percentage of students scoring one, two or three points on the Farm Drone task. Most students (ranging from 76 to 92 percent) were able to receive as least one point for this task. Scoring at least one point meant that these students were able to complete all of the objectives although perhaps not efficiently. A smaller percentage of students scored three points (ranging from 16 to 40 percent).
Table 2. Percentage of students scoring one, two or three points on the Farm Drone task, by education system, 2018
| Education system | Percentage scoring one out of three points | Percentage scoring two out of three points | Percentage scoring three points |
| Denmark† 1 | 92 (0.5) | 83 (0.9) | 26 (1.2) |
| Finland | 87 (1.0) | 80 (1.2) | 29 (1.0) |
| France | 87 (0.8) | 77 (1.0) | 40 (1.3) |
| Germany | 83 (1.2) | 73 (1.2) | 18 (1.2) |
| Korea, Republic of | 90 (0.8) | 86 (1.0) | 39 (1.9) |
| Luxembourg | 76 (0.5) | 66 (0.5) | 16 (0.3) |
| Portugal††1 | 88 (0.8) | 78 (1.1) | 20 (1.1) |
| Not meeting sample participation requirements | |||
| United States | 86 (0.6) | 77 (0.7) | 34 (1.1) |
| Benchmarking participant meeting sample participation requirements | |||
| North Rhine-Westphalia (Germany) | 84 (0.9) | 73 (1.3) | 17 (1.1) |
Notes: Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. Reproduction of Figure 4.1. from Preparing for Life in a Digital World international report (2019), p. 117
† Met guidelines for sampling participation rates only after replacement schools were included.
†† Nearly met guidelines for sampling participation rates after replacement schools were included.
1 National defined population covers 90% to 95% of the national target population.
Afterthoughts
This author took the Farm Drone test. I ran my first program in the 1970s from a terminal connected to a large campus computer center and spent thirty years learning and forgetting various programming languages and applications. I thought that I might whiz through the task, which was fun and engaging, but I was not accustomed to visual programming, and it took a moment to acclimate to the visual code blocks. Moreover, I solved the problems so much more slowly than a younger member of my household who also completed the task. They grew up gaming. It was no contest.
I am looking forward to examining more released ICILS items, testing my performance, and learning how 8th graders perform on the upcoming 2028 ICILS assessments. I am expecting students, on average, to be more accustomed to the CT tasks of ICILS, given the increasing emphasis on education technology.
What did you think of this sample test? How did you do? If you enjoyed it, there are three other items for you to try out.
- Computer and Information Literacy (CIL) is defined as “an individual’s ability to use computers to investigate, create, and communicate in order to participate effectively at home, at school, in the workplace, and in society.” Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2013). Preparing for Life in a Digital Age:The IEA International Computer and Information Literacy Study International Report, Springer Cham., p. 17. https://link.springer.com/book/10.1007/978-3-319-14222-7 ↩︎
- Computational Thinking (CT) is defined as an individual’s ability to recognize aspects of real-world problems which are appropriate for computational formulation and to evaluate and develop algorithmic solutions to those problems so that the solutions could be operationalized with a computer. Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2019). Preparing for Life in a Digital Age:The IEA International Computer and Information Literacy Study 2018 International Report. Springer Cham., p. 27. https://link.springer.com/book/10.1007/978-3-030-38781-5 ↩︎
- J. Fraillon (ed.) (2025). An International Perspective on Digital Literacy: Results from ICILS 2023. Springer Cham., p. 134. https://link.springer.com/book/10.1007/978-3-031-87722-3 ↩︎
- T. Herzog, The Farmer was Replaced (online game). Retrieved on February 8, 2026 from https://store.steampowered.com/app/2060160/The_Farmer_Was_Replaced/ ↩︎
- Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., & Friedman, T. (2019). IEA International Computer and Information Literacy Study 2018 Assessment Framework. Amsterdam, The Netherlands: International Association for the Evaluation of Educational Achievement (IEA), p. 44. Retrieved on October 29, 2025 from https://www.iea.nl/publications/assessment-framework/iea-international-computer-and-information-literacy-study-2018 ↩︎
- For ICILS 2023, the scale labels and boundaries were revised using 2023 CT items and final proficiency levels were set. Ibid., pp. 136-141. ↩︎
- There are no upper or lower bounds to the CT scale. More detailed information can be found in J. Fraillon (ed.) (2025). An International Perspective on Digital Literacy: Results from ICILS 2023. Springer Cham. pp. 113-114. https://link.springer.com/book/10.1007/978-3-031-87722-3 ↩︎
- Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., and Friedman, T. (2019). IEA International Computer and Information Literacy Study 2018 Assessment Framework. Amsterdam, The Netherlands: International Association for the Evaluation of Educational Achievement (IEA), pp. 34-35. ↩︎
- J. Fraillon (ed.) (2025). An International Perspective on Digital Literacy: Results from ICILS 2023. Springer Cham. pp. 344-352. https://link.springer.com/book/10.1007/978-3-031-87722-3 ↩︎
Leave a Reply