Welcome to this research experiment evaluating the approach and infrastructure surrounding two distinct approaches towards progress grading and feedback in practical cyber security exercises.

Each player will be presented with two sets of similar practical computer security exercises to work through, each using a distinct grading approach. After each challenge you will answer questions about your experience during the practical exercise, with a focus on the grading and feedback approach. The intent is for you to solve all 4 challenges on both scenarios. Hints and supporting materials will be available to aid you in solving the challenges if you require them. Collect a hints and tips printout in class (or if you are participating remotely view the hints and tips here: https://docs.google.com/document/d/1iFQE_v6lsG0ojYtEr-fRgOcIYN5olcTEl4EiGBsty2Q/edit#heading=h.sl7nogjb29ee).

Here is a link to the survey: https://forms.gle/stFqfSYoqZ6vx8ZJA

The order of activities is:

• Complete the four challenges in the first scenario you were instructed to do.

• Open the survey link above and complete the likert scale questions in section 3 - 5 of this survey. Do not answer the open-ended questions in section 7 the first time you take the survey.

• Complete the four challenges in the second scenario you were instructed to do.

• Take the survey a second time, this time answering all questions in the survey including section 7.

All participants that complete the challenges and both post-challenge surveys are automatically entered into a prize draw to win one of three amazon gift vouchers (up to £40 in value). The draw will take place once the event concludes and winners will be announced via email.

This is an optional extra curricular event which will give you the opportunity to complete some practical CTF challenges and provide your perspective on the different grading and feedback approaches taken.

This research is intended to benefit others: results will be published in academic venues, such as peer-reviewed conferences and journals. Results will also be summarised and disseminated in presentations.

Data from this event will be gathered and analysed in research aimed at understanding and improving the technologies surrounding practical cyber security exercises. Data will be gathered based on system usage and participant feedback. Data collected includes usage data collected from the systems used during the exercises and questionnaire responses. All data gathered will be anonymised prior to analysis and publication.

By taking part in this study you confirm that you: have read and understand the above; understand that all personal information will remain confidential and that all efforts will be made to ensure you cannot be identified (except as might be required by law); agree that data gathered in this study may be stored anonymously and securely, and may be used for future research; understand that participation is voluntary and that you are free to withdraw at any time without giving a reason; and you agree to take part in this study. If you do choose to withdraw please contact Thomas Shaw via email (address below).

This project has received ethical approval from Leeds Beckett University.

For more information about the CTF event and this research, please contact Thomas Shaw (Thomas.Shaw@leedsbeckett.ac.uk), Lecturer in Cyber Security and Ph.D. Candidate, Leeds Beckett University.