whyrating-engine-legacy/urt-taxonomy/track-a-training/A1-Annotator-Quickstart.md

A1: Annotator Quickstart Guide

Universal Review Taxonomy (URT) v5.1

Purpose: Quick reference for human annotators to classify review spans Version: 5.1 | Status: Production Ready | Date: 2026-01-23

---

Table of Contents

1. Quick Decision Tree
2. Domain Summary Table
3. Category Quick Reference (28 Categories)
   • 3.5 High-Frequency Subcodes
4. Metadata Cheat Sheet
5. Common Mistakes to Avoid
6. V vs R Rule: Scam vs Rip-off
7. J4 vs R3 Rule: Process vs Ownership
8. Span Boundary Rules
9. Worked Examples
10. USN Quick Reference
11. Billing/Returns/Refunds Decision Table
12. Data Privacy/Security Decision Table
13. CR Trigger Word Reference
14. Causal Chain Quick Guide
15. Inter-Annotator Agreement Rules

---

1. Quick Decision Tree

Use this flowchart for every span. Go top-to-bottom; first "YES" wins.

START: Read the span
        |
        v
+-----------------------------------------------+
| Is it about the THING they bought/received?   |
| (product, service, treatment, outcome)        |
+-----------------------------------------------+
        | YES --> O (Offering)
        | NO  |
              v
+-----------------------------------------------+
| Is it about HOW PEOPLE behaved?               |
| (staff attitude, skill, communication)        |
+-----------------------------------------------+
        | YES --> P (People)
        | NO  |
              v
+-----------------------------------------------+
| Is it about TIME, STEPS, or FRICTION?         |
| (waiting, process, ease, resolution steps)    |
+-----------------------------------------------+
        | YES --> J (Journey)
        | NO  |
              v
+-----------------------------------------------+
| Is it about the SPACE or INTERFACE?           |
| (physical place, app/website, ambiance)       |
+-----------------------------------------------+
        | YES --> E (Environment)
        | NO  |
              v
+-----------------------------------------------+
| Is it about ABILITY TO ACCESS or INCLUSION?   |
| (availability, disability, language, bias)    |
+-----------------------------------------------+
        | YES --> A (Access)
        | NO  |
              v
+-----------------------------------------------+
| Is it about PRICE, COST, or WORTH?            |
| (money, effort, value, transparency)          |
+-----------------------------------------------+
        | YES --> V (Value)
        | NO  |
              v
+-----------------------------------------------+
| Is it about TRUST, PATTERNS, or LOYALTY?      |
| (honesty over time, brand, recovery, bond)    |
+-----------------------------------------------+
        | YES --> R (Relationship)
        | NO  |
              v
    RE-READ SPAN -- may need splitting

Memory Aid: O-P-J-E-A-V-R = "Offer People a Journey in an Environment with Access to Value and Relationship"

---

2. Domain Summary Table

Code	Domain	Core Question	One-Line Definition	Example (+)	Example (-)
O	Offering	Does it work?	The core product/service delivered	"Perfectly cooked steak"	"Phone won't charge"
P	People	How did they treat me?	Human interactions and behavior	"So helpful and patient"	"Rude and dismissive"
J	Journey	Was it smooth?	Process, timing, friction	"Seated immediately"	"Waited 45 minutes"
E	Environment	Is the space okay?	Physical, digital, ambient context	"Beautiful clean space"	"App keeps crashing"
A	Access	Can I get it?	Availability, accessibility, inclusion	"Open 24/7"	"No wheelchair ramp"
V	Value	Is it worth it?	Cost, pricing, and worth	"Great bang for buck"	"Total rip-off"
R	Relationship	Can I trust them?	Trust, reliability, loyalty	"Always reliable"	"Shady company"

Default Owner Routing

Domain	Default Owner
O	Product / Operations
P	HR / Training
J	Operations / Process
E	Facilities / IT
A	Compliance / Design
V	Finance / Pricing
R	Leadership / CX

---

3. Category Quick Reference

28 Categories - 4 per domain. Use this for URT-Core or to narrow down to subcodes.

O - Offering (4 categories, 18 subcodes)

Code	Category	Definition
O1	Function	Does it do what it's supposed to do?
O2	Quality	How well is it made or executed?
O3	Completeness	Is everything included that should be?
O4	Fit	Does it match customer's specific needs?

P - People (4 categories, 20 subcodes)

Code	Category	Definition
P1	Attitude	Disposition, manner, emotional tone of staff
P2	Competence	Knowledge, skill, professional capability
P3	Responsiveness	Attentiveness, initiative, follow-through
P4	Communication	Quality of information exchange

J - Journey (4 categories, 20 subcodes)

Code	Category	Definition
J1	Timing	Speed, punctuality, time management
J2	Ease	Effort required and friction encountered
J3	Reliability	Consistency and predictability of process
J4	Resolution	How problems are handled when they arise

E - Environment (4 categories, 20 subcodes)

Code	Category	Definition
E1	Physical Space	Tangible environment attributes
E2	Digital Space	Online and application interface
E3	Ambiance	Intangible environmental qualities (mood, noise)
E4	Safety	Security and wellbeing factors

A - Access (4 categories, 20 subcodes)

Code	Category	Definition
A1	Availability	Can you get it when you need it?
A2	Accessibility	Can everyone use it regardless of ability?
A3	Inclusivity	Does it work for diverse backgrounds?
A4	Convenience	Is it easy to reach and engage with?

V - Value (4 categories, 20 subcodes)

Code	Category	Definition
V1	Price	The monetary cost itself
V2	Transparency	Clarity and honesty about costs
V3	Effort	Non-monetary costs (time, hassle, mental load)
V4	Worth	Overall value assessment

R - Relationship (4 categories, 20 subcodes)

Code	Category	Definition
R1	Integrity	Honesty and ethical behavior
R2	Dependability	Consistency over time
R3	Recovery	Response to failures (ownership, apology, compensation)
R4	Loyalty	Investment in ongoing relationship

---

3.5 High-Frequency Subcodes Quick Reference

These are the most commonly used subcodes across all domains. Memorize these first.

Offering (O) - Top Subcodes

Code	Name	Definition	+ Example	- Example
O1.01	Works/Doesn't Work	Basic functionality	"Software runs perfectly"	"Car won't start"
O1.02	Performance Level	How well it operates	"Incredibly fast"	"Sluggish and laggy"
O1.05	Outcome Achievement	Did customer accomplish goal?	"Passed my exam!"	"Treatment didn't work"
O2.01	Materials/Inputs	Quality of components	"Real leather"	"Cheap plastic parts"
O2.03	Presentation	Visual/aesthetic quality	"Gorgeous plating"	"Looked thrown together"
O2.05	Condition at Delivery	State when received	"Still warm"	"Arrived damaged"
O4.01	Specification Match	Matches what was ordered	"Exactly right"	"Wrong size delivered"

People (P) - Top Subcodes

Code	Name	Definition	+ Example	- Example
P1.01	Warmth/Friendliness	Approachability	"So welcoming"	"Cold and unfriendly"
P1.02	Respect	Treating with dignity	"Made me feel valued"	"Talked down to me"
P1.03	Empathy	Understanding situation	"Really got my frustration"	"Couldn't care less"
P2.01	Knowledge	Understanding of products	"Knew every detail"	"Couldn't answer basics"
P2.03	Problem-Solving	Ability to address issues	"Found creative solution"	"Just said 'can't'"
P3.01	Attentiveness	Awareness of needs	"Always checking in"	"Had to flag them down"
P3.03	Availability	Present when needed	"Easy to find someone"	"Impossible to reach"
P4.01	Clarity	Understandable info	"Explained clearly"	"Confusing jargon"

Journey (J) - Top Subcodes

Code	Name	Definition	+ Example	- Example
J1.01	Wait Time	Time spent waiting	"Seated immediately"	"45 min past appointment"
J1.02	Service Speed	Time for delivery	"Next day delivery"	"Took three weeks"
J1.03	Response Time	Time to address inquiries	"Replied in minutes"	"Days for a response"
J2.01	Simplicity	Straightforward processes	"So easy to do"	"Needlessly complicated"
J3.02	Accuracy	Correct execution	"Order exactly right"	"Wrong items delivered"
J4.02	Resolution Process	How problems handled	"Clear escalation path"	"Transferred in circles"
J4.03	Resolution Speed	Time to fix	"Fixed same day"	"Took weeks to resolve"

Environment (E) - Top Subcodes

Code	Name	Definition	+ Example	- Example
E1.01	Cleanliness	Hygiene and tidiness	"Spotless facilities"	"Filthy bathrooms"
E1.02	Maintenance	Condition and upkeep	"Everything works"	"Broken equipment"
E2.01	Interface Design	Visual/interaction quality	"Beautiful app"	"Cluttered mess"
E2.02	Functionality	Features working	"Everything works"	"Buttons broken"
E3.01	Atmosphere/Vibe	Overall mood	"Calm and relaxing"	"Stressful chaos"
E4.01	Physical Safety	Protection from harm	"Felt completely safe"	"Dangerous conditions"

Access (A) - Top Subcodes

Code	Name	Definition	+ Example	- Example
A1.01	Operating Hours	When accessible	"Open 24/7"	"Banker's hours only"
A1.02	Booking Access	Ability to schedule	"Easy online booking"	"3 weeks for appointment"
A1.03	Inventory/Capacity	Available	"Always in stock"	"Perpetually sold out"
A2.01	Physical Accessibility	Mobility accommodations	"Wheelchair accessible"	"No ramps or elevators"
A3.01	Language Support	Multiple languages	"Staff spoke my language"	"English only, no help"
A4.01	Location	Physical accessibility	"Convenient location"	"Middle of nowhere"
A4.02	Parking	Vehicle accommodation	"Easy free parking"	"Parking nightmare"

Value (V) - Top Subcodes

Code	Name	Definition	+ Example	- Example
V1.01	Absolute Price	The actual cost	"Very affordable"	"Outrageously expensive"
V1.04	Hidden Costs	Unexpected charges	"No hidden fees"	"Surprise $50 charge"
V2.01	Pricing Clarity	Understanding costs	"Clear price list"	"Impossible to understand"
V2.03	Advertising Accuracy	Marketing matches reality	"As advertised"	"Bait and switch"
V2.04	Terms Fairness	Policy reasonableness	"Fair cancellation"	"Predatory contract"
V4.01	Overall Value	Total assessment	"Excellent value"	"Total rip-off"
V4.02	Quality-Price Ratio	What you get for payment	"Great for the price"	"Pay more, get less"

Relationship (R) - Top Subcodes

Code	Name	Definition	+ Example	- Example
R1.01	Truthfulness	Accurate representations	"Everything as stated"	"They flat out lied"
R1.02	Promise Keeping	Honoring commitments	"Always deliver on word"	"Never keep promises"
R1.04	Ethics	Moral business conduct	"Ethical company"	"Shady business practices"
R2.02	Consistency	Same experience over time	"Always great"	"Varies wildly"
R3.01	Acknowledgment	Admitting failures	"Owned their mistake"	"Denied any wrongdoing"
R3.03	Compensation	Making amends	"More than made up for it"	"Offered nothing"
R4.01	Recognition	Acknowledging repeat customers	"They remember me"	"Stranger every time"

---

4. Metadata Cheat Sheet

7 dimensions, 24 total values. Profile determines which are required.

4.1 Valence (V) - ALWAYS REQUIRED

Code	Label	When to Use	Markers
V+	Positive	Praise, satisfaction	"great," "loved," "excellent"
V-	Negative	Complaint, dissatisfaction	"terrible," "hated," "worst"
V0	Neutral	Factual observation	"they have," "it was"
V+-	Mixed	Both positive and negative in same span	"good but," "despite," "however"

Rule: Prefer splitting over V+- unless the positive and negative target the same thing.

4.2 Intensity (I) - ALWAYS REQUIRED

Code	Label	When to Use	Markers
I1	Mild	Slight preference/concern	"a bit," "somewhat," "could be better"
I2	Moderate	Clear but not extreme	Standard adjectives, no intensifiers
I3	Strong	Emphatic, intense	"extremely," "worst ever," CAPS, !!!, profanity

4.3 Specificity (S) - Standard+

Code	Label	When to Use	Example
S1	Vague	General impression only	"Service was bad"
S2	Moderate	Some details or context	"Service was slow at dinner"
S3	Specific	Names, times, amounts	"Waiter John took 40 mins at 7pm Saturday"

4.4 Actionability (A) - Standard+

Code	Label	When to Use	Example
A1	Low	Feeling, no clear action path	"I just didn't like it"
A2	Medium	Suggests improvement area	"Food could be warmer"
A3	High	Specific implementable fix	"West bathroom stall lock is broken"

Note: S and A are correlated but not identical. "John was rude" = S3 (named person) but A2 (action unclear).

4.5 Temporal Reference (T) - Standard+

Code	Label	When to Use
TC	Current	This specific visit/experience (default)
TR	Recent	Recent pattern of experiences
TH	Historical	Long-standing pattern
TF	Future	Expectations or predictions

4.6 Evidence Type (E) - Standard+

Code	Label	When to Use	Constraint
ES	Stated	Explicitly said by customer	Text directly says it (default)
EI	Inferred	Logically entailed by text	Must be directly deducible, not speculative
EC	Contextual	Requires surrounding text	Span uses "that," "they," "it" referencing earlier

EI Rule: Only use when text logically entails the inference.

• OK: "Took 3 weeks to reply" = EI for slow response time
• NOT OK: "Seemed tired" = Cannot infer fatigue without more evidence

4.7 Comparative Reference (CR) - Standard+

Code	Label	When to Use	Trigger Words
CR-N	None	No comparison to past (DEFAULT - 90% of spans)	(no temporal language)
CR-B	Better	Explicit improvement	"better now," "improved," "finally fixed"
CR-W	Worse	Explicit decline	"worse now," "used to be good," "gone downhill"
CR-S	Same	Explicitly unchanged	"still," "as always," "same as before"

CR Rules:

1. Only assign CR-B/W/S when customer explicitly compares to their own past
2. CR-N is default -- only change when comparison is explicit
3. Self-comparison only (not competitor comparison)
4. Implicit decline counts: "used to be good" = CR-W

Metadata by Profile

Dimension	Lite	Core	Standard	Full
Valence	Required	Required	Required	Required
Intensity	Optional	Required	Required	Required
Specificity	--	--	Required	Required
Actionability	--	--	Required	Required
Temporal	--	Optional	Required	Required
Evidence	--	--	Required	Required
Comparative	--	Optional	Required	Required

---

5. Common Mistakes to Avoid

Top 10 Disambiguation Pitfalls

#	Mistake	Correct Approach
1	Confusing O1.01 (Works/Doesn't Work) with J3.03 (System Uptime)	O1.01 = product function. J3.03 = system/service availability. "Car won't start" = O1.01. "Website was down" = J3.03.
2	Confusing P3.04 (Follow-Through) with R1.02 (Promise Keeping)	P3.04 = specific interaction ("said they'd call back, didn't"). R1.02 = trust/pattern ("they never keep their word").
3	Confusing J2.01 (Simplicity) with E2.04 (Navigation)	J2 = effort/friction in process ("too many steps"). E2 = qualities of interface ("buttons don't work").
4	Confusing J3.01 (Consistency) with O1.04 (Reliability)	J3.01 = process delivers same result each time. O1.04 = product itself works consistently. "Pizza is always good" = J3.01. "Phone always connects" = O1.04.
5	Confusing E2.03 (Interface Performance) with O1.02 (Product Performance)	E2.03 = the app/interface is slow. O1.02 = the core product performs poorly ("car accelerates slowly").
6	Confusing A3.05 (Equal Treatment) with P1.02 (Respect)	A3.05 = identity-based discrimination ("treated differently because of X"). P1.02 = general disrespect without identity framing.
7	Confusing V2.xx with R1.xx	V2 = pricing/terms/advertising specifically. R1 = trust/integrity/organizational character framing.
8	Over-splitting cause-effect spans	"Long wait because understaffed" = single span. Primary = J1.01, Secondary = A1.04. Only split if different sentences or genuinely different issues.
9	Assigning CR-B/W/S without explicit comparison	CR-N is default. "Service is slow" = CR-N. "Service is still slow" = CR-S. Must have explicit temporal language.
10	Inventing causal codes	Causal chain only when text explicitly states or logically entails. "Probably a training issue" = annotator speculation = NO causal code.

Quick Disambiguation Tests

Question	If YES	If NO
Does it mention the product/service itself?	Likely O	Check P-R
Does it name or describe a person?	Likely P	Check J-R
Does it mention time, steps, or waiting?	Likely J	Check E-R
Does it describe a place or interface?	Likely E	Check A-R
Does it mention accessibility or availability?	Likely A	Check V-R
Does it mention price, cost, or worth?	Likely V	Check R
Does it mention trust, patterns, or "always/never"?	Likely R	Re-read span

---

6. V vs R Rule: Scam vs Rip-off

This is the most common confusion. Use this anchor:

The Rule

Feedback Type	Domain	Rationale
"This is a rip-off"	V4.01	Complaint about the exchange (poor value)
"This company is a scam"	R1.04	Moral judgment about intent (they're dishonest)

Criminal Metaphor Mapping

Word/Phrase	Domain	Reasoning
"scam"	R1.04	Character judgment
"fraud"	R1.04	Character judgment
"crooks"	R1.04	Character judgment
"rip-off"	V4.01	Exchange complaint
"robbery"	V4.01	Exchange complaint
"highway robbery"	V4.01	Exchange complaint
"overpriced"	V1.01 or V4.02	Price/value complaint

Decision Heuristic

Ask: Is the complaint about...

• What I got vs. what I paid? --> V (Value)
• Their intent to deceive/harm me? --> R (Relationship)

Examples

Span	Primary	Rationale
"Total scam company"	R1.04	Moral judgment about intent
"They charged me twice"	J3.02	Process accuracy issue
"refused to refund"	R3.03	Recovery/compensation failure
"This is robbery"	V4.01	Exchange complaint
"They deliberately mislead people"	R1.01	Organizational truthfulness
"Overpriced garbage"	V4.02	Quality-price ratio

---

7. J4 vs R3 Rule: Process vs Ownership

Both J4 and R3 deal with problems, but from different angles.

The Distinction

Domain	Focus	Key Question
J4 (Resolution)	What they did to fix it	How was the problem handled operationally?
R3 (Recovery)	How they took responsibility	Did they own it, apologize, make amends?

Category-Level Comparison

J4 Category	R3 Category	Distinction
J4.01 Problem Acknowledgment	R3.01 Admitting Failures	J4.01 = recognized the issue exists. R3.01 = admitted wrongdoing/fault.
J4.02 Resolution Process	--	J4.02 = the operational steps taken
J4.03 Resolution Speed	--	J4.03 = how fast they fixed it
J4.04 Resolution Quality	R3.03 Compensation	J4.04 = was fix adequate? R3.03 = did they make amends beyond fixing?
J4.05 Prevention	R3.04 Improvement	J4.05 = process change. R3.04 = organizational commitment to change.

Decision Heuristic

Ask: Is the feedback about...

• The mechanics of the fix? --> J4 (Journey: Resolution)
• Their accountability and making things right? --> R3 (Relationship: Recovery)

Examples

Span	Primary	Rationale
"They immediately sent a replacement"	J4.02 or R3.03	If emphasizing speed/process = J4. If emphasizing making amends = R3.
"Took weeks to resolve"	J4.03	Resolution speed
"They owned their mistake"	R3.01	Admitting fault
"Sincere apology"	R3.02	Expression of regret
"Offered nothing"	R3.03	Compensation failure
"Changed their process"	J4.05	Prevention action
"Blamed me instead"	R3.05	Ownership failure

---

8. Span Boundary Rules

When to SPLIT

1. At contrasting conjunctions: but, however, although, despite, yet
2. When subject changes: location -> service -> food
3. When valence changes: positive -> negative
4. When domain changes: different aspect of experience
5. When in different sentences (unless tightly linked cause-effect)

When to KEEP TOGETHER

1. Cause-effect in same clause: "Long wait because understaffed"
   • Primary = impact (J1.01 Wait Time)
   • Secondary = cause (A1.04 Staffing)
2. Same target, mixed assessment: "Good but overpriced" (same product)
3. Elaboration of same point: "Service was slow. Had to wait 40 minutes."

Max Spans Guidance

Sentence Type	Typical Spans
Simple sentence	1-2 spans
Complex sentence	2-3 spans
If tempted to create 4+	Re-read; probably over-splitting

Cause-Effect Examples

Span	Treatment	Primary	Secondary
"Long wait because understaffed"	Single	J1.01	A1.04
"Food cold, had to send it back"	Single	O2.05	J4.02
"App crashed so I lost my data"	Single	E2.02	E4.03
"Rude waiter. Also the food was cold."	TWO spans	P1.02 / O2.05	--
"Support was slow because the system kept crashing"	Single	J1.03	J3.03

V+- vs Split Decision

Feedback	Action
"Great product, terrible onboarding"	SPLIT: two different targets
"Good but overpriced"	KEEP: same target, use V+-
"Nice staff despite the chaos"	SPLIT if chaos is separate feedback

Test: If the positive and negative parts have different domain targets, split them.

---

9. Worked Examples

Example 1: Restaurant Review (Multiple Spans)

"The pasta was perfectly al dente and beautifully plated, but our waiter disappeared for 20 minutes and we had to flag down someone else to get the check. Good value for a nice dinner out."

Span	Primary	Secondary	V	I	S	A	T	E	CR
"pasta was perfectly al dente"	O1.02	--	V+	I2	S2	A2	TC	ES	CR-N
"beautifully plated"	O2.03	--	V+	I2	S2	A2	TC	ES	CR-N
"waiter disappeared for 20 minutes"	P3.03	--	V-	I2	S3	A3	TC	ES	CR-N
"flag down someone else"	P3.01	--	V-	I2	S2	A2	TC	ES	CR-N
"Good value for nice dinner"	V4.01	--	V+	I1	S1	A1	TC	ES	CR-N

Example 2: Software with Improvement Signal (CR-B)

"Implementation took twice as long as promised, but once we went live, the system has been rock solid. Much more stable than the last version."

Span	Primary	Secondary	V	I	CR	Note
"Implementation took twice as long"	J1.02	R1.02	V-	I3	CR-N	No self-comparison
"system has been rock solid"	O1.04	--	V+	I3	CR-N	Current state
"Much more stable than last version"	O1.04	--	V+	I2	CR-B	Explicit improvement

Example 3: Persistent Problem (CR-S)

"Still waiting forever for appointments. Third time I've complained about this and nothing has changed."

Span	Primary	Secondary	V	I	CR	Note
"Still waiting forever"	A1.02	J1.01	V-	I3	CR-S	"Still" = unchanged
"Third time I've complained"	J4.05	R3.04	V-	I2	CR-S	Pattern, unchanged

Causal Chain (Full profile only):

• MG-O (Oversight failure) -- Evidence: EI
• Justification: "Third time" + "nothing changed" logically entails oversight failure

Example 4: Quality Decline (CR-W)

"This used to be my favorite restaurant but the quality has really gone downhill since they changed chefs."

Span	Primary	Secondary	V	I	CR	Note
"used to be my favorite"	R2.02	--	V-	I2	CR-W	Past-positive implies decline
"quality has really gone downhill"	O2.02	--	V-	I3	CR-W	Explicit decline

Example 5: Service Recovery (CR-B)

"My package arrived damaged, which was frustrating. But they immediately sent a replacement and it arrived perfect. Great recovery!"

Span	Primary	Secondary	V	I	CR
"package arrived damaged"	O2.05	--	V-	I2	CR-N
"immediately sent replacement"	R3.03	J4.03	V+	I2	CR-N
"arrived perfect"	O2.05	--	V+	I2	CR-B
"Great recovery"	R3	--	V+	I2	CR-N

Example 6: Scam vs Rip-off (V vs R)

"Total scam company. They charged me twice and refused to refund. This is robbery."

Span	Primary	Secondary	V	I	CR
"Total scam company"	R1.04	--	V-	I3	CR-N
"charged me twice"	J3.02	--	V-	I2	CR-N
"refused to refund"	R3.03	V2.04	V-	I2	CR-N
"This is robbery"	V4.01	--	V-	I3	CR-N

Example 7: Accessibility Issue

"No wheelchair ramp and the staff didn't seem to care."

Span	Primary	Secondary	V	I	S	A	T	E	CR
"No wheelchair ramp"	A2.01	--	V-	I2	S3	A3	TC	ES	CR-N
"staff didn't seem to care"	P1.03	--	V-	I2	S1	A2	TC	EI	CR-N

Example 8: Digital Environment

"App is beautiful but crashes constantly. Finally works on mobile though!"

Span	Primary	Secondary	V	I	CR
"App is beautiful"	E2.01	--	V+	I2	CR-N
"crashes constantly"	E2.02	--	V-	I3	CR-N
"Finally works on mobile"	E2.05	--	V+	I2	CR-B

Example 9: Cause-Effect (Single Span)

"Support was painfully slow because their system kept going down."

Span	Primary	Secondary	V	I	Note
Full span	J1.03	J3.03	V-	I3	Impact (slow) + cause (system down)

Example 10: Multi-Domain Complex Review

"Great location and the room was clean, but checkout took forever and they tried to charge me for a minibar I never touched. Classic hotel scam."

Span	Primary	Secondary	V	I	CR	Note
"Great location"	A4.01	--	V+	I2	CR-N	Convenience
"room was clean"	E1.01	--	V+	I2	CR-N	Physical space
"checkout took forever"	J1.01	--	V-	I3	CR-N	Wait time
"tried to charge me for minibar I never touched"	V1.04	J3.02	V-	I2	CR-N	Hidden cost + accuracy
"Classic hotel scam"	R1.04	--	V-	I3	CR-N	Character judgment

---

10. USN Quick Reference

URT String Notation for compact tagging.

Format

URT:{Profile}:{codes}:{Valence}{Intensity}[:{metadata}][:{causal}]

Profiles

Code	Profile	Code Tier
L	Lite	Domain (O, P, J...)
C	Core	Category (O1, P2...)
S	Standard	Subcode (O1.01, P2.03...)
F	Full	Subcode + Causal

Quick Examples

Review	USN
"Food was cold" (Lite)	URT:L:O:-2
"Food was cold" (Core)	URT:C:O2:-2
"Food was cold, like last time" (Standard)	URT:S:O2.05:-2:22TR.ES.S
"Rude staff and unfair policy" (Standard)	URT:S:P1.02+V2.04:-3:22TC.ES.N
"Much better now!" (Standard)	URT:S:V4.01:+2:11TC.ES.B

Metadata Encoding (Standard+)

{Specificity}{Actionability}{Temporal}.{Evidence}.{Comparative}

Example: 22TC.ES.N
         ^^ = S2, A2
           ^^ = TC (current)
              ^^ = ES (stated)
                 ^ = CR-N (no comparison)

Valence Symbols

Symbol	Meaning
+	Positive
-	Negative
0	Neutral
+-	Mixed

---

Quick Checklist Before Submitting

1. One primary code per span (no exceptions)
2. Max 2 secondary codes (prevents over-tagging)
3. Checked "Don't Confuse With" in spec for ambiguous codes
4. Used decision tree for domain selection
5. Split at domain changes (different aspects = different spans)
6. CR-N unless explicit comparison (default is no comparison)
7. No invented causes (causal chain only when text supports)
8. Verified intensity markers (I3 requires emphatic language)
9. Cause-effect kept together (impact = primary, cause = secondary)
10. V vs R correctly applied (exchange = V, character = R)

---

11. Billing/Returns/Refunds Decision Table

Special guidance for common billing and refund scenarios.

Feedback Type	Primary Code	Secondary	Rationale
"Returns process was easy"	J4.02	--	Resolution process ease
"Returns process was a nightmare"	J4.02	--	Resolution process friction
"Refund took too long"	J4.03	--	Resolution speed
"Refund policy is unfair"	V2.04	--	Terms fairness
"They refused to refund me" (trust framing)	R3.03	V2.04	Compensation failure + policy
"Wrong amount refunded"	J3.02	V1.04	Accuracy + hidden costs
"They honored the warranty"	R2.05	--	Guarantee honor
"Worthless guarantee"	R2.05	--	Guarantee failure
"Charged me twice"	J3.02	--	Process accuracy
"Hidden fees at checkout"	V1.04	V2.02	Hidden costs + fee disclosure

---

12. Data Privacy/Security Decision Table

Feedback Type	Primary Code	Secondary	Rationale
"Security incident occurred"	E4.03	--	Security breach
"I don't trust them with my data"	R1.03 or R1.04	E4.03	Trust + security
"Data breach notification"	E4.03	R1.03	Security + transparency
"Privacy policy concerns"	V2.04	R1.04	Terms + ethics
"Account was hacked"	E4.03	--	Security failure
"They sell my data"	R1.04	V2.04	Ethics + terms

---

13. CR Trigger Word Reference

Quick lookup for Comparative Reference assignment.

CR-B (Better) - Improvement Signals

Trigger Phrase	Example
"better now"	"Service is much better now"
"improved"	"Quality has improved"
"finally fixed"	"They finally fixed the app!"
"much faster/cleaner/friendlier than before"	"Much faster than my last visit"
"they've turned it around"	"They've really turned things around"
"glad they addressed it"	"Glad they addressed the issue"
"not like last time" (positive context)	"Not like last time - actually worked!"

CR-W (Worse) - Decline Signals

Trigger Phrase	Example
"worse now"	"Quality is worse now"
"used to be good"	"This used to be my favorite restaurant"
"has gone downhill"	"Really gone downhill lately"
"not what it used to be"	"Just not what it used to be"
"declining"	"Quality is declining"
"deteriorating"	"Service has been deteriorating"
"they've really fallen off"	"They've really fallen off"
"was great before"	"Was great before the renovation"
"no longer what it was"	"No longer the place I loved"

CR-S (Same) - Persistence Signals

Trigger Phrase	Example
"still"	"Service is still slow"
"as always"	"Great as always"
"same as before"	"Same problems as before"
"nothing has changed"	"Complained twice, nothing has changed"
"yet again"	"Wrong order yet again"
"once again"	"Once again disappointed"
"as usual"	"Late as usual"
"same old"	"Same old problems"

CR-N (None) - No Comparison

• Default for ~90% of spans
• First-time customer reviews
• Competitor comparisons ("better than Brand X")
• No temporal language present

---

14. Causal Chain Quick Guide (Full Profile Only)

Only assign causal codes when text explicitly states or logically entails the cause. When in doubt, omit.

Conditions Layer (CD-)

Code	Name	When to Use	Example Text
CD-S	Staff State	Fatigue, training, motivation stated	"Server seemed exhausted"
CD-T	Team Dynamics	Handoffs, coordination issues stated	"They kept passing me around"
CD-E	Equipment	Malfunction stated	"The machine was broken"
CD-F	Facility	Maintenance, capacity stated	"Place was falling apart"
CD-O	Operational	Understaffing, demand surge stated	"Clearly understaffed"

Management Layer (MG-)

Code	Name	When to Use	Example Text
MG-P	Planning	Scheduling, forecasting failure stated	"No one scheduled for weekend"
MG-T	Training	Training gap stated	"Clearly not trained properly"
MG-O	Oversight	Supervision failure inferable	"Third time reporting this" (EI)
MG-R	Resources	Resource allocation stated	"No supplies available"
MG-C	Communication	Policy relay failure stated	"Staff didn't know the policy"

Systemic Layer (SY-)

Code	Name	When to Use	Example Text
SY-R	Resource Decisions	Budget, investment stated	"Cheap owners"
SY-P	Policy/Procedure	Bureaucracy, rules stated	"Company policy is the problem"
SY-C	Culture	Values, priorities stated	"Management doesn't care"
SY-S	Standards	Quality thresholds stated	"They've lowered their standards"
SY-H	Human Capital	Compensation, hiring stated	"Can't keep good employees"
SY-X	External	Market, regulatory stated	"Ever since COVID"

Causal Chain Validity Test

Review Text	Causal Code?	Valid?	Why
"Long wait because they were short-staffed"	CD-O	Yes	Explicitly stated
"The machine was broken"	CD-E	Yes	Explicitly stated
"Fifth time I've reported this"	MG-O	Yes	Logically entailed (EI)
"They seem understaffed"	--	No	"Seem" = speculation
"Probably a training issue"	--	No	Annotator speculation
"Management doesn't care" (stated by customer)	SY-C	Yes	Customer explicitly states

---

15. Inter-Annotator Agreement Rules

Mandatory Conventions

1. One primary code per span - no exceptions
2. Max 2 secondary codes - prevents over-tagging
3. Max 3 spans per sentence - if more, reconsider boundaries
4. Prefer splitting over V+- - unless truly inseparable
5. Check "Don't Confuse With" - before finalizing subcode
6. Evidence type required - must justify EI inferences
7. No invented causes - causal_chain only when text supports
8. CR-N is default - only use CR-B/W/S when comparison is explicit

Tie-Break Rules

When unsure between two codes:

Disagreement Type	Resolution
Different domains	Use decision tree; first "YES" wins
Same category, different subcodes	Prefer more specific subcode
Different intensity	Defer to linguistic markers (see I3 markers)
Different actionability	A3 requires specific implementable action
Different CR	CR-N unless explicit temporal language present

Calibration Check Questions

Before submitting, ask yourself:

1. Would another annotator choose the same primary domain?
2. Is the secondary code genuinely distinct (different category)?
3. Is intensity justified by linguistic markers?
4. Is CR assignment supported by explicit comparison language?
5. Would I assign causal codes if I didn't know the industry?

---

Where to Find More

Document	Purpose
URT-Specification-v5.1.md	Full specification with all 140 subcodes
URT Issue Lifecycle Framework	Analytics layer for resolution tracking
URT QA Protocol	Calibration and audit procedures

---

Appendix: Profile Requirements Summary

URT-Lite (Micro-business, quick triage)

Required: primary_code (domain), valence
Optional: intensity
Forbidden: secondary_codes, subcodes, S/A/E/CR/causal

URT-Core (Small business, dashboards)

Required: primary_code (category), valence, intensity
Optional: secondary_codes (max 2, categories only), temporal, comparative
Forbidden: subcodes, S/A/E, causal

URT-Standard (Operations, routing, analytics)

Required: primary_code (subcode), valence, intensity, specificity,
          actionability, temporal, evidence, comparative
Optional: secondary_codes (max 2, subcodes only), causal_chain, confidence
Forbidden: domains, categories as primary

URT-Full (Enterprise, root cause, audit)

Required: All Standard fields + causal_chain (when evidence supports),
          linked_spans, confidence, annotator_notes
Optional: None - all fields should be completed
Forbidden: domains, categories as primary

---

URT v5.1 Annotator Quickstart Guide | Track A: Training Materials