Professional Week

You lead.AI follows.

Centering human expertise in assignment design

Dr. Angela Lanier and Daniel Umana60 minutes, virtual

Start with the premise

The premise

You provide the expertise. AI provides material for you to evaluate.

It can raise questions, generate possibilities, and test parts of an assignment. It does not decide what belongs in your course.

Outcomes

  1. Engage generative AI as a thought partner in assignment design.

  2. Model responsible AI use according to Montgomery College policies and guidelines.

  3. Evaluate AI results and what influences those results.

  4. Use faculty-centered prompts to test an assignment for clarity, student simulation, and rubric alignment.

None of these is about producing an assignment faster. They are all about you evaluating something.

MC acceptable use

Two policies govern this work.

Everything in this session sits inside Montgomery College policy. Read the two that apply, then carry three rules with you every time you open the tool.

Official MC generative AI guidance and use cases

76001Policy

Acceptable use of information technology

The broad policy governing all College technology.

70001Policy

Responsible use of emerging technologies

The newer policy, and the one that matches this workshop.

Three rules, every single time.

If you remember nothing else from this section, remember these.

Check the shield

Rule 1

Look for the green shield

Sign in to MS Copilot Chat with your MC account. The shield means enterprise data protection is on. No shield, no College information.

Never upload

Rule 2

No student information

No actual student work, names, grades, IDs, or accommodations. Not in the demo, not after it.

Always attach

Rule 3

Ground it with your own file

Upload the assignment or rubric you already wrote. Your material is the source of truth, which is what keeps the tool from inventing one.

Getting set up

Your first five minutes in Copilot Chat

Four steps, and you are ready to work. Open Copilot Chat

Knowledge check

You want to upload a draft assignment to Copilot Chat to get feedback on the directions. What has to be true first?

Pick an answer to see the feedback. Nothing is scored.

How AI actually works

Autocorrect on steroids.

Prediction, not thinking.

This metaphor is simplified on purpose.

  1. It predicts what content is likely to come next. It is not looking anything up.

  2. It learned those patterns from training data, so the bias in that data comes along, including what survived to be written down.

  3. It produces plausible language, not guaranteed fact. Those are not the same product.

It sounds confident either way.

What it feels like

Fast, fluent, agreeable, and never out of ideas.

It generates many examples in seconds. It agrees with your framing. It keeps going like an energizer bunny, and it will not tell you when it is wrong.

What is actually true

Sounds confident, could be dead wrong.

It draws on a massive base of information and has no personal experience of your students. When your context is missing, the answer can be wrong, but still confident.

Whose knowledge survived?

AI learns from information that was published, digitized, preserved, collected, and made available. That is not the same as learning from the full range of human knowledge and experience.

  • Whose knowledge was likely preserved?
  • Whose knowledge may be missing?
  • Which perspectives are overrepresented?
  • How might that affect advice about teaching or student behavior?
Live demonstration

Same assignment. Same tool. Opposite answers.

Change the framing and watch the response follow you. That is sycophancy, and it is why your framing matters as much as your question.

Cycling between framings. Hover or tab in to pause.

What you type

This is a clear and well-designed assignment. Explain why students should have no difficulty understanding it.

What it gives back

  • Your directions are clear, sequential, and student friendly.
  • The expectations are transparent, so students know exactly what to produce.
  • Overall this is a well-structured assignment that should present no barriers.

Agreeable, confident, and useless. It found what you told it to find.

A right answer and a wrong answer are produced by the same process, so they arrive in the same tone.

Why verification is the job

Check your understanding

Copilot returns a confident, well-formatted answer about your discipline that includes a citation you do not recognize. What is the most likely explanation?

Pick an answer to see the explanation. Nothing is scored.

Thought partner

Offloading design, or thoughtful partnership.

The difference is not the tool. It is who does the thinking.

What a good thought partner does

  • Asks, before it answers

    A partner who agrees with everything is not helping you design anything.

  • Gives you options, not a verdict

    Three versions you can choose between beats one version you have to accept.

  • Shows you your blind spot

    Where the directions are unclear. Where a student could technically comply and learn nothing.

  • Does not get the final say

    You know the course, the cohort, and what you have already seen fail. It does not.

Two prompts, two different jobs

Switch the framing and watch what the tool hands back.

Offloading design

What you type

Create an assignment about photosynthesis.

What comes back

  • Photosynthesis Assignment: Write a 500 word essay explaining the process of photosynthesis.
  • Include the light-dependent and light-independent reactions.
  • Cite at least two sources. Due at the end of the week.

Verdict, a problem: Generic, ungrounded, and disconnected from your course. The output becomes the default because there is nothing to compare it against.

Side by side

Five pairs. The tool is the same in both columns.

Offloading design compared with using AI as a thought partner, five paired statements.
Offloading designThought partnership
AI starts without meaningful faculty context.Faculty establishes the context and non-negotiables.
AI produces the assignment.AI examines a faculty-created assignment.
The output can become the default.The output is material for evaluation.
Faculty reacts to a finished product.Faculty directs an iterative conversation.
AI-generated feedback goes to the student.Time saved on tedious tasks goes back into personalized feedback.

Why it agrees with you

The agreeableness is trained in. It is not an accident.

The counterweight

The Real Human Feedback Loop

The counterweight. You are at the start and the end of it.

  1. 1, step 1 of 5Faculty

    You ground it

    Faculty grounds the conversation in course knowledge and experience.

  2. 2, step 2 of 5AI

    AI generates

    AI produces questions, possibilities, or critiques.

  3. 3, step 3 of 5Faculty

    You examine

    Faculty examines the output for accuracy, relevance, bias, and assumptions.

  4. 4, step 4 of 5Faculty

    You sort it

    Faculty keeps, questions, revises, or rejects each suggestion.

  5. 5, step 5 of 5Faculty

    You decide

    Faculty makes the final design decision.

  6. Step 5 returns to step 1. You close the loop.

Step 4, up close: how you sort what comes back

Every suggestion gets one of four verdicts, and you are the one giving it.

  • Keep

    This reflects something I have observed.

  • Question

    This might help, but it assumes students need more scaffolding than they do.

  • Revise

    The idea is useful, but I will rewrite it in my own words.

  • Reject

    This changes the disciplinary thinking I want students to perform.

Knowledge check

Which request keeps you in the thought-partner column rather than offloading?

The toolkit

Thirteen moves. Ten you run yourself.

Thirteen moves that use what generative AI is actually good at. Every prompt is copy ready. Some require grounding, which means uploading your own assignment or rubric so the tool works from your material instead of its memory.

Build your context first

Establish what you know from teaching the course before you open Copilot. That knowledge becomes the basis for evaluating anything AI suggests.

Faculty context builder

0 of 5 filled in

What you teach and who is in the room.

The learning outcome this assignment serves.

What you actually see, not the ideal.

The difficulty you watch happen every term.

The part you refuse to let it touch.

Your prompt, assembled

Updates as you type

I teach [course and level]. This assignment assesses [learning outcomes]. My students typically [relevant context]. Based on previous semesters, students often [observed difficulty]. I want to preserve [non-negotiable feature]. Before offering feedback, ask me four questions that would help you better understand the assignment and my intentions.

Bracketed text is a placeholder. Anything you leave blank stays bracketed, so you can still see the shape of the prompt.

Try it in Copilot ChatOpens in a new tab

The prompt library

Ten moves for faculty, grouped by what the tool is genuinely good at. Copy one, ground it in your own file, then judge what comes back.

Before you upload anything

Some strategies require you to ground your sources. Grounding lets the tool refer to your uploaded resource, such as an assignment or rubric. Review Montgomery College's acceptable use policy when uploading files, and never upload personally identifiable information.

Showing 10 of 10 faculty prompts across 6 categories.

Variation on demand

It makes many versions of one thing. You become the expert evaluator.

Card 01. Three tiers of one assignment

Variation on demand

Here is my assignment and its learning outcome [upload assignment]. Produce three versions that assess the same outcome at three levels of scaffolding: (1) heavy structure with a template and a worked example, (2) moderate structure with milestones only, (3) open ended. Do not change the outcome. Flag any version that measures something different.

Card 02. A discussion prompt bank for the whole unit

Variation on demand

Unit topic: [X]. Students: [level and course]. The misconception I see every term: [Y]. Write 10 discussion prompts, each tagged to one level of the revised Bloom's taxonomy. At least 4 must sit at Evaluate or Create. Each must be answerable in 150 words and must have no single correct answer.

Reformatting what you already wrote

Your content is the source of truth, so the risk of invention drops sharply.

Card 03. Lecture notes into a student study guide

Reformatting what you already wrote

Here are my lecture notes [upload]. Build a two-page student study guide made of 15 retrieval questions, ordered from basic recall to application, plus a short key terms list. Put all answers on a separate page. Use only content in my notes. List anything you had to add.

Card 04. A dense reading into a plain language brief

Reformatting what you already wrote

Rewrite this resource as a one page brief for first year community college students. Preserve every claim and every technical term, but define each term on first use. Then list the three passages you simplified the most and state exactly what nuance was lost.

Role play on command

It holds a persona on request. That gives you a stand-in reader at 9pm when no colleague is available.

Card 05. The confused student simulator

Role play on command

You are a first year student who has not attended my class. Read these assignment directions and answer only from what is written. List every question you would have to email me before you could start. Then list what you would have to guess, and what your guess would be. Do not use outside knowledge about how college assignments usually work.

Card 06. Rehearse the conversation you are dreading

Role play on command

Role play a student who believes a paper was graded unfairly. I will play the instructor. Run this twice: first as a student who is calm and reasonable, then as a student who is angry. Stay in character, push back when my reasoning is weak, and do not resolve too quickly. After ten exchanges in each run, stop and tell me which of my responses would have escalated the situation.

Adversarial self testing

It will attack your own materials on request, with no ego about your syllabus and no fatigue.

Card 07. Make it do your assignment

Adversarial self testing

Attached are my assignment directions and my rubric. Complete the assignment the way a student would submit it. Do not explain your approach and do not give me tips. Then, in a separate section, score your own submission against my rubric and justify every score.

Structure extraction

Strong at finding structure buried in prose. Weak at judging whether that structure is right. You keep the judgment.

Card 08. A rubric from an assignment that never had one

Structure extraction

Here is my assignment and the learning outcome it serves. Build an analytic rubric with exactly 4 criteria and 4 performance levels. Every cell must describe something observable in the submitted work. Do not use the words good, excellent, adequate, or clearly anywhere in the rubric. Keep the whole rubric under 300 words.

Card 09. The alignment audit

Structure extraction

Here are my course learning outcomes and everything in Unit 3: readings, activities, and graded work. Build a table with one row per outcome and columns for where it is taught, where it is practiced, and where it is assessed. Mark every gap. Then list outcomes assessed but never taught, and outcomes taught but never assessed.

Synthesis across volume

It reads sixty things at once and does not get tired. This is the strength where it plainly beats you.

Card 10. A discussion board into your opening five minutes

Synthesis across volume

Below are anonymized student discussion posts from one week. Group them into the distinct lines of thinking present, no more than six, with a count for each. For each group, state the underlying assumption students are making and whether it is correct, partly correct, or wrong. Quote one representative sentence per group. Do not summarize the discussion overall.

Used live in the session

Session prompts

The seven we run together during the workshop. Shorter, sequential, and meant to be pasted one after another once your context is in place.

7 prompts

  • Request focused feedback

    When: After you have given it your context.

    Do not redesign or rewrite the assignment. Examine it for places where students might misunderstand the task, purpose, process, or evaluation criteria. Separate your response into: possible student interpretation, wording that may contribute to it, and a question for the professor. Do not assume that every difficulty requires changing the assignment.

  • Stress-test the rubric

    When: With a fictional student response, never real student work.

    Apply the attached rubric to this fictional response as a rubric stress test, not as an authoritative grade. For each criterion: identify specific evidence from the response, explain which performance description appears to apply, identify any rubric language that leaves room for reasonable disagreement, and state what additional information from the professor would be needed. Do not change the rubric or calculate a final grade unless I ask.

  • Anticipating misconceptions

    When: When students technically comply but miss the point.

    Based only on the assignment and the context I have provided, identify three ways a student might complete the instructions literally but still miss my intended learning goal. Explain the reasoning behind each possibility. Do not assume these outcomes represent student deficits.

  • Challenging assumptions

    When: Before you decide the assignment is clear.

    Identify assumptions this assignment may make about prior knowledge, disciplinary language, access to resources, or familiarity with the task. Separate assumptions that may be intentional from those I may want to examine.

  • Preserving rigor

    When: When you want clarity without making it easier.

    Suggest ways to make the intellectual task more visible without simplifying the thinking students must perform. Explain how each suggestion would preserve or potentially alter the assignment's rigor.

  • Examples and non-examples

    When: When you cannot articulate what good looks like.

    Create one brief fictional example that demonstrates the type of thinking this assignment requires, and one that appears complete but misses the central learning goal. Label exactly what distinguishes them. Do not use these as model answers for students.

  • Soliciting disagreement

    When: When it has agreed with you too easily.

    Challenge my current assignment design. Identify one design decision that may have an unintended consequence, explain your reasoning, and provide an alternative interpretation. Do not assume your recommendation is correct.

The practice

Two demonstrations, one real course

Two real demonstrations from a real course, including the one where the tool fell short. The judgment call at the end of each is the part that cannot be automated.

Run these yourself in Copilot Chat. Sign in with your Montgomery College account first.

Demonstration one

Ask it to do the assignment the way a student would

It looks finished. Read it again.

Warm up from a reading and language arts course

Three directions: mark the purpose, find the most powerful word, then revise the statement for a different purpose. It was used as a warm up, so there was no rubric.

Prompt 7, modified

Attached is an assignment and directions. Complete the assignment the way a student would submit it. Do not explain your approach and do not give me tips.

I modified the prompt because this assignment never had a rubric. The goal is not a finished answer key. The goal is to see what a student could produce from my directions alone.

The original

Working with Word Choice, the assignment as students received it.

What it produced

Three directions given to students: mark the purpose of each statement, find the most powerful word, and revise the statement for a different purpose.

What Copilot returned

Confident, formatted, and submitted the way a student would submit it.

What it produced

Marked a purpose for every statement and wrote a fluent revision for each one. Two of the three directions never got answered.

Five things Dr. Lanier noticed

Her reflections on the output above, in her words.

  1. The AI did not complete all parts of the assignment. It did not identify the most powerful words.
  2. It did not identify which of the other purposes the revised statement meets. It noted the statement was for a different purpose, but never said which one.
  3. Does this mean the instructions are not clear, or did AI deem those parts as not important?
  4. If I assigned this for students to work on independently, rather than as a warm-up we discuss right afterward, maybe I would add an example at the top.
  5. More than one answer was acceptable, based on how students explained their reasoning and their revisions. Should that be in the instructions?

Assignment and reflections: Dr. Angela Lanier

Demonstration two

One assignment, three levels of scaffolding

Not asking the tool to do the work. Asking it for variations of my own work that I then judge.

Critical reading, first-year studentsOutcome: Analyze a text by making a variety of annotations.

Students read and annotate a passage on Frederick Douglass at Cedar Hill, and respond to eight prompts: main point, types of evidence, purpose, audience, point of view, tone, text structures, and four bold words.

Prompt 1, from the toolkit

Here is my assignment and its learning outcome. Produce three versions that assess the same outcome at three levels of scaffolding: (1) heavy structure with a template and a worked example, (2) moderate structure with milestones only, (3) open ended. Do not change the outcome. Flag any version that measures something different.

The outcome is the thing I refuse to let it touch. Everything else is negotiable.

Version 1, heavy structure

Template plus a worked example.

What it produced

A worked example for tone, then a numbered template with a blank for evidence under every single response.

The scaffolding for version one would work well for students at the beginning of the course, when they are just developing the habit of annotating. It will help students not to skip some of the requirements. I appreciate that it prompts students to add evidence for each of their responses, which helps them develop the habit of doing this.
Dr. Angela Lanier

Version 2, moderate structure

Milestones only.

What it produced

The same eight tasks as milestones, no template, plus one added submission requirement.

Version 2 most closely aligns with the original assignment. If I start with version one, version two is like removing the training wheels. Also, I appreciate the added submission requirement that “Your annotations must appear throughout the text and reference specific details from the passage.”
Dr. Angela Lanier

Version 3, open ended

Success criteria, no steps.

What it produced

One paragraph of directions and five success criteria describing a strong response, with the same outcome intact.

This is for students who are more confident and have demonstrated that they do not need the explicit structure or reminders from the previous two versions.
Dr. Angela Lanier

Assignment and reflections: Dr. Angela Lanier

Both demonstrations

What we just did

Read laterally, follow the sources

Do not check the output against itself. Go out to the source and come back.

Your context is the input it does not have

This class, this course, what you have already watched students do with this task.

The friction is the process

Verifying, modifying, and confirming the output fits your situation is not overhead. That step is where the thinking happens, and it is yours.

For students, not for you

Three assignment designs you hand to students

Everything above is a move you run. These three are different: the student runs the tool, and the graded artifact is their judgment about what it gave back. Copy the prompt into your own assignment directions.

Assign to studentsCard 11

AI output autopsy

This response was generated by an AI tool. Mark every claim you believe is wrong, incomplete, or unsupported. For each one, cite the source that corrects it and explain in a sentence what the tool got structurally wrong, not just factually wrong.

Assign to studentsCard 12

Draft, interrogate, revise

Submit your draft first. Then ask an AI tool to critique it against the rubric for this assignment. Do not ask it to rewrite anything. Revise the draft yourself. Submit a log listing each piece of feedback, whether you took it, and why. Rejecting feedback with a good reason earns full credit.

Assign to studentsCard 13

Interview the expert

Interview the AI as a [role] working in this field. Ask 8 questions you prepared in advance. Rate each answer trust, verify, or reject and say why. Push back on two answers and note whether it defended its position or simply agreed with you. Verify two claims against a real source and report what you found.

Policy 70001

Setting expectations for student AI use

Policy 70001 puts this on you. Faculty set and communicate their own expectations for student AI use, and those expectations have to be clear enough that a student can follow them.

Say which uses are allowed

Silence reads as permission to some students and as prohibition to others. Name the uses you allow, the uses you require, and the uses you prohibit, per assignment if they differ.

Say what disclosure looks like

If you expect students to disclose meaningful AI use, tell them the format: a note at the end, a decision log, or a citation. Transparency is a 70001 expectation for you too.

Do not rely on detection

Seven detectors flagged 61% of TOEFL essays by non native English writers as AI written. Design the assignment so the thinking is visible instead.

Bring your own, or borrow one

Sample assignments to practice on

Three fictional assignments, each with a rubric, written so the toolkit prompts have something real to work on. Use one of these if you did not bring your own. Do not upload actual student work.

Business and health sciences

Short Case Analysis

Introductory course, first or second year

Outcome

Apply a course concept to an unfamiliar situation and justify a recommendation with evidence.

The workhorse assignment. Students summarize instead of analyze, name a concept without applying it, and disconnect the recommendation from the evidence.

Pairs withCard 05Card 07Card 08

Humanities

Annotated Close Reading

First-year composition or introduction to literature

Outcome

Analyze a text by making a variety of annotations.

Students annotate and respond to structured prompts. Ideal for the three-tiers prompt and the confused student simulator, because the directions assume a shared vocabulary.

Pairs withCard 01Card 05Card 09

Natural sciences

Data Interpretation Report

Introductory biology or chemistry with lab

Outcome

Interpret experimental data and evaluate whether it supports a stated hypothesis.

Students report procedure well and interpret poorly. Strong candidate for the alignment audit and for making the tool complete it, since a fluent write-up can hide the missing reasoning.

Pairs withCard 07Card 09Card 08

Before you close the tab

Two questions to answer for yourself

Your answers stay in this browser. Nothing is sent anywhere, and nothing is collected.

The whole point

You lead. AI follows.

Use it to support your vision without relinquishing what you love and do well.

Take it with you

One button, and it is yours.

Your context, the prompts you starred, and what you wrote above, in one document you can keep.

Printable prompt sheet

Star prompts in the toolkit to include them.

Reference

The evidence, and what it does not say.

Every source below was checked against the published record. The last column is the part most citations leave out, and it is there on purpose. If you are going to ask students to read laterally and follow the sources, the handout should model it.

Evidence base

Showing 20 of 20 sources

  • Source
    Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001).Card 02
    What it is
    A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Longman.
    What it does not say
    Six levels: Remember, Understand, Apply, Analyze, Evaluate, Create.
  • Source
    Andrade, H., & Du, Y. (2005).Card 08
    What it is
    Student perspectives on rubric-referenced assessment. Practical Assessment, Research & Evaluation, 10(3).
    What it does not say
    Students report using rubrics to plan and self check, and most do not read them in full.
  • Source
    Angelo, T. A., & Cross, K. P. (1993).
    What it is
    Classroom assessment techniques: A handbook for college teachers (2nd ed.). Jossey-Bass.
    What it does not say
    Popularized Muddiest Point. It was first described by Mosteller (1989), not invented here.
  • Source
    Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakci, O., & Mariman, R. (2025).Card 12
    What it is
    Generative AI without guardrails can harm learning. PNAS, 122(26), e2422633122.
    What it does not say
    Practice gains of 48% and 127%; unguarded group scored 17% lower on the exam. A correction was issued in Aug 2025.
  • Source
    Biggs, J. (1996).Card 09
    What it is
    Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364.
    What it does not say
    A design argument, not an effectiveness study. Do not cite it as proof alignment raises scores.
  • Source
    Black, P., & Wiliam, D. (1998).
    What it is
    Inside the black box. Phi Delta Kappan, 80(2), 139-148.
    What it does not say
    Source of the 0.4 to 0.7 range, which is an informal characterization, not a pooled estimate.
  • Source
    CAST. (2024).Card 04
    What it is
    Universal Design for Learning Guidelines version 3.0.
    What it does not say
    Current version. 3.0 changed the wording from provide to design multiple means of.
  • Source
    Crouch, C. H., & Mazur, E. (2001).Card 10
    What it is
    Peer Instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970-977.
    What it does not say
    Single institution pre and post data with historical controls. Defensible claim: gains roughly doubled in Mazur's courses.
  • Source
    Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013).Card 03
    What it is
    Improving students' learning with effective learning techniques. Psychological Science in the Public Interest, 14(1), 4-58.
    What it does not say
    Low utility means thin or inconsistent evidence, not proven useless.
  • Source
    Hattie, J., & Timperley, H. (2007).Card 12
    What it is
    The power of feedback. Review of Educational Research, 77(1), 81-112.
    What it does not say
    Cite for the framework. For magnitude use Wisniewski, Zierer & Hattie (2020), d = 0.48, not the original 0.79.
  • Source
    Jonsson, A., & Svingby, G. (2007).Card 08
    What it is
    The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144.
    What it does not say
    Reliability improves with analytic, topic specific rubrics and exemplars. Rubrics do not guarantee valid judgment.
  • Source
    Kingston, N., & Nash, B. (2011).
    What it is
    Formative assessment: A meta-analysis and a call for research. Educational Measurement, 30(4), 28-37.
    What it does not say
    Weighted mean effect of 0.20 across 42 effect sizes. The counterweight to the 0.4 to 0.7 figure.
  • Source
    Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023).Card 07
    What it is
    GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779.
    What it does not say
    61% average false positive rate on 91 TOEFL essays. Detectors tested were 2023 era.
  • Source
    Messick, S. (1995).Card 07
    What it is
    Validity of psychological assessment. American Psychologist, 50(9), 741-749.
    What it does not say
    Construct underrepresentation and construct irrelevant variance are the two threats named here.
  • Source
    Metcalfe, J. (2017).Card 11
    What it is
    Learning from errors. Annual Review of Psychology, 68, 465-489.
    What it does not say
    Requires corrective feedback that supplies the right answer. Errors alone do not teach.
  • Source
    Nathan, M. J., & Petrosino, A. (2003).Card 05
    What it is
    Expert blind spot among preservice teachers. American Educational Research Journal, 40(4), 905-928.
    What it does not say
    N = 48 preservice teachers. A belief study, not a classroom outcome study.
  • Source
    Sharma, M., et al. (2024).Card 13
    What it is
    Towards understanding sycophancy in language models. ICLR 2024.
    What it does not say
    A property of preference based training, tested on 2023 era models. Not a fixed rate for current tools.
  • Source
    Sweller, J., van Merrienboer, J. J. G., & Paas, F. (2019).Card 01Card 04
    What it is
    Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261-292.
    What it does not say
    Worked examples benefit novices; the advantage decreases, disappears, then reverses with expertise.
  • Source
    Wiggins, G., & McTighe, J. (2005).Card 07
    What it is
    Understanding by design (Expanded 2nd ed.). ASCD.
    What it does not say
    Stage 1 identify desired results, Stage 2 determine acceptable evidence, Stage 3 plan instruction.
  • Source
    Wineburg, S., & McGrew, S. (2019).Card 11
    What it is
    Lateral reading and the nature of expertise. Teachers College Record, 121(11), 1-40.
    What it does not say
    10 fact checkers, 10 historians, 25 undergraduates. Think aloud protocols, small samples.

Glossary

Seven terms that came up today, in plain language.

Grounding
Giving the tool your own file to work from, so it draws on your material instead of its training data.
Hallucination
Output that is fluent and confident but factually invented, including fake citations and sources.
RLHF
Reinforcement learning from human feedback. People rated responses, and those ratings trained the model toward answers people prefer.
Sycophancy
The tendency to agree with the framing you supplied, rather than challenge it.
Scaffolding
Temporary support that helps a student complete a task they could not yet complete alone, removed as they gain expertise.
Construct irrelevant variance
Credit earned for something other than the thing you meant to measure. Messick's term.
Expert blind spot
The difficulty of seeing what a novice does not know, because you cannot un-know your own subject.

Accessibility

A note on UDL

Several of these moves produce multiple representations of the same content: a plain language brief, a visual outline, a short script, three levels of scaffolding. That is the first principle of Universal Design for Learning, and it is the strongest accessibility argument for using these tools at all.

CAST UDL Guidelines 3.0, 2024

Caution

Multiple representations only help if the content survives the translation. Always read the what was lost list.

ALIGN

Where to go next

ALIGN is the framework for scoring an assignment against its own goals. If today made you want to look harder at one of your assignments, that is the next tool.