Judging Engineering Notebooks

This article covers how to review and score an Engineering Notebook at an event: what you receive, how to work through a notebook efficiently, how notebook scores feed award selection, and how to handle the judgment calls that come up. It is written for judges and Judge Advisors.

For the rubric itself and what each section covers, see Engineering Notebook Rubrics. For interviews, see Judging Team Interviews. For how notebook and interview findings come together in award selection, see Judging Deliberations.

Definitions of the terms used here are maintained in the central glossary.

What You Receive

Teams submit an Engineering Notebook along with three short summaries: a Season Summary, a Code Summary, and a Credit Summary. Notebooks may be paper or digital, and both are scored against the same criteria.

The summaries are navigation tools, not additional submissions to evaluate:

• Season Summary — the team's one-page account of its season, pointing to the notebook entries behind each highlight. Read this first.
• Code Summary — how the team's code works, written to be followed by someone who does not read code. Teams that use no code do not submit one.
• Credit Summary — a single list of the outside sources currently used on the robot and in the code, gathered from where they appear in the notebook and the code.

The summaries are not scored. A thin or missing summary is not a deduction. Use them to spend your time on the parts of the season the team most wants you to see. For what teams are told about them, see Season, Code, and Credit Summaries for Judging.

Reviewing Engineering Notebooks

The Rubric

Score every criterion (every row) on the rubric. A submitted notebook scores at least 1 on each, so the lowest possible total is 16 and the highest is 64.

• Engineering Notebook Rubric – Letter Size
• Engineering Notebook Rubric – A4 Size

What You Are Looking For

You are reading for evidence that a team went through the Engineering Design Process — that it defined a problem, tried things, learned from what happened, and improved. The rubric breaks that into sixteen criteria so scoring stays consistent between judges, but the thing you are looking for is the process, not for specific words from the Engineering Design Process (Define, Develop Solutions, Optimize) to appear on the page.

That process may not look tidy on the page. Every team documents differently. A team that tested something, got a confusing result, tried again, and eventually changed direction is doing engineering well, even if the entries are messy.

Teams also work at very different scales. A first-year VEX IQ team with two elementary school members is going through the same process as an experienced V5 team, at a different level of sophistication. Score what the team in front of you did against the criteria, not against the best notebook you saw today.

What the Scores Mean

• Proficient means the team did the thing. It is a good score. A notebook scoring Proficient across the board totals 48 of 64 — a solid record of a real season.
• Exemplary is not simply Proficient done well. Each criterion draws a specific line, usually the team going one step past the question asked: explaining what it chose against rather than only what it chose, or citing a measurement rather than saying it tested. If a team did not cross that particular line, Proficient is the accurate score, not a penalty.
• Developing and Beginning describe what is missing, not how hard the team worked. A team can pour a season into its robot and still score Beginning on testing because it did not write its tests down.

Working Through a Notebook

1. Start with the Season Summary and follow its pointers into the notebook. A notebook can run past a hundred pages, and the summary exists so you do not have to find the substance yourself.
2. Score every criterion on the rubric. A submitted notebook scores at least 1 on each, so the lowest possible total is 16 and the highest is 64.

Three practices keep scoring consistent across a judging team:

• Apply the same time limit to every notebook. Decide it before you start and hold to it. A team should not be advantaged by being reviewed early in the day.
• Have at least two judges review each notebook where staffing allows. Compare scores before ranking rather than averaging blindly, and talk through any criterion where you disagree by more than one level.
• Remember, you are not an investigator, and finding problems is not part of your job. The rubric already accounts for thin records: a notebook missing testing, iteration, or evidence of work across the season will score accordingly without any special handling.

Use the Credit Summary when you reach the criterion on crediting outside sources. It tells you what to expect to find in the notebook, which is faster than searching.

Using Notebook Scores for Awards

The score exists to rank a pile of notebooks so the conversation can start. It is not a grade on the team's season. Rubric scores stay within the judging team — teams do not receive them, and they are not shared with coaches, Event Partners, or other volunteers.

• Once every notebook is scored, rank the pile and identify the strongest group — at a typical event, roughly the top ten. That group is the pool you consider for the Design, Innovate, and Excellence Awards. 
• A score of 48 or above is a useful guide to determine if the notebook is Fully Developed. It means the notebook averaged Proficient across all 16 criteria. Where the cut falls at your event depends on what was submitted, so treat 48 as a benchmark rather than a rule.
• Many awards include the criterion, "Fully Developed Engineering Notebook." Fully Developed means that this notebook is one of your top notebooks of the event and includes examples of each category of the Engineering Design Process (Define, Develop Solutions, Optimize).

Final rankings within that pool come from discussion, not from scores alone. Two notebooks a single point apart are not meaningfully different, and the conversation about which team's record best shows its engineering is the part that matters. See Judging Deliberations.

Many Judged Awards requires a notebook submission. For what each award recognizes and their criteria, see Award Descriptions.

What Does Not Affect a Score

• Length, appearance, and graphic design. A legible working notebook scores as well as a polished one.
• An unusually polished notebook with no sign of work in progress should score lower on "Record spans the season," not higher.
• Format. Paper and digital are scored identically.
• Handwriting, spelling, and grammar, except where they make an entry impossible to read.
• Accommodations. A notebook produced with assistive technology, scribing, or additional processing time is scored on the same criteria as any other.

Ownership and Honesty

Students write the notebook. No adult and no generative AI tool writes it for them.

Generative AI counts as an outside source. A team may use an AI tool to learn something, and must credit any idea or code that came from one. What a team may not do is have an AI tool write its notebook or its code. A notebook that credits an AI-derived idea is doing the right thing.

Three situations that are legitimate and should not count against a team:

• Official VEX starting points. A team may compete with a Hero Bot as built, or use a VEXcode template unchanged. These are credited as starting points, and building from one does not make the work less the team's own.
• Credited collaboration. Teams are encouraged to work with and learn from each other. A team that names another team as a source is following the rules, not admitting a violation.
• Similar designs. Two teams can arrive at the same solution independently, particularly within one game. Resemblance is not evidence of copying.

Purchased work is different. Work a team bought, traded for, or was given rather than doing itself cannot become the team's own, no matter how well it is credited. Buying and trading parts is fine; buying or being handed finished work is not.

If something genuinely concerns you — including any doubt about whether the work is the team's own — tell the Judge Advisor and let them handle it. Do not express a concern as a lower score. Then go back to scoring what is on the page.

Confidentiality

Everything you see and record during notebook judging stays within the judging team.

• Notebook contents. Do not discuss a team's notebook outside the judging team, and do not share images or copies.
• Rubric scores and completed rubrics. These are working documents for the judging team. They are not returned to teams, shown to coaches, or shared outside the judging room.
• Digital notebooks. Delete any downloaded copies at the conclusion of the event.

If a coach or team asks how their notebook scored, refer them to the Judge Advisor.