Responsible AI is not a feature.
It's the architecture.
ClearPath AI is built on a single conviction: a confident wrong answer is more dangerous than no answer at all. When vulnerable people seek help with food, housing, or mental health, an AI that confidently directs them to a non-existent program doesn't just waste time — it erodes trust in the entire help-seeking process. Our Responsible AI Framework is not an afterthought. It is the architectural foundation of the system.
What we will never compromise on
These are not aspirational guidelines. They are architectural constraints baked into every layer of the system — enforced in code, not just in documentation.
Confidence scores are ALWAYS visible
Every single result shows a calibrated confidence percentage. Not a vague "match found" — a specific number derived from the BART-large-MNLI zero-shot classification scores, dampened for known over-classification categories. When we say 73% match, we mean it. When we say 45%, that is our honest assessment. Users deserve to know how much weight to give a recommendation before they invest their time pursuing it.
Crisis detection is ALWAYS hardcoded
We never trust AI to detect crisis signals. Our crisis layer uses exact-match and regex patterns against a curated keyword list covering suicidal ideation, self-harm, domestic violence, and substance abuse emergencies. When a crisis keyword is detected, the AI is bypassed entirely — no classification, no confidence scoring, no hallucinated advice. Immediate connection to 988 Suicide & Crisis Lifeline and 211.org. Reliability over elegance when lives are at stake.
Human escalation at <70% confidence
When our model confidence drops below 70%, we do not try to be clever. We ask clarification questions. When it drops below 50%, we escalate to a human navigator at 211.org. This threshold is not arbitrary — it reflects the point at which zero-shot classification accuracy degrades meaningfully for community resource categorization. A confident wrong answer is more dangerous than no answer at all, especially for someone seeking help with food, housing, or mental health.
We NEVER auto-contact services on behalf of users
ClearPath AI will never dial a phone number, submit an application, or send a message on behalf of a user without explicit consent. We provide information and connections; the user takes action. This is a deliberate architectural boundary. Auto-contacting services could lead to unintended commitments, privacy violations, or inappropriate service engagement. We are a navigator, not an agent — we show the path, but the user walks it.
We NEVER store PII without consent
Account creation is optional — no email or password required for guest use. For guests, session data is purged when the browser closes. Authenticated users can access their history across sessions. No Google Analytics, no Facebook Pixel, no third-party tracking. The only data that leaves our system is the classification API call to HuggingFace, which processes text without storing it. Users seeking help for domestic violence or substance abuse often do so from shared devices — they deserve absolute privacy by default.
Known limitations are ALWAYS documented
We explicitly list every known failure mode: misclassification of ambiguous queries, over-classification into mental health, missed crisis signals from non-standard phrasing, model API downtime, and confidence miscalibration. We do not hide behind vague disclaimers — we name specific risks, assess their likelihood and impact, and detail the mitigation for each. A team that cannot articulate its system's limitations has not thought deeply enough about responsible deployment.
What could go wrong
Every AI system has failure modes. The responsible approach is not to pretend they don't exist — it's to name them, assess them, and build mitigations for each one. Here are the five risks we take most seriously, and exactly how we address them.
What we couldn't fix
These limitations are not weaknesses in our Responsible AI framework — they are strengths. A team that cannot articulate its system's limitations has not thought deeply enough about responsible deployment. We believe honesty about what we cannot do is as important as demonstrating what we can.
We can't verify real-time resource availability
A food bank listed as open may have changed hours. A shelter may be at capacity tonight. We show "Last verified" dates and encourage users to call ahead, but we cannot confirm real-time availability. In a production system, this would require API integrations with each resource provider — an effort beyond our hackathon scope but architecturally planned.
We can't guarantee accuracy of third-party data
Our resource database is curated from verified sources, but those sources can contain errors. A government database may list incorrect eligibility requirements. A nonprofit's listed phone number may be outdated. We mitigate this by showing data provenance and encouraging verification, but we cannot independently verify every data point. This is why "call to confirm" is always visible.
We can't detect crisis in all languages
Our hardcoded crisis keyword list covers English-language crisis expressions. We acknowledge that non-English speakers, code-switchers, and users of African American English (AAE) may not be served by our current keyword list. This is a documented, honest limitation. We mitigate by always offering human navigator access and by expanding our keyword list through community feedback.
We can't replace professional judgment
ClearPath AI is a navigator, not a counselor, social worker, or healthcare provider. We help people find resources — we do not diagnose, prescribe, or make decisions. When a user's situation requires professional expertise, we connect them to a trained 211 navigator who can provide it. Our confidence scores and human escalation thresholds are designed to route complex cases to humans, not to replace them.
Testing for fairness, not assuming it
We conduct systematic bias testing across demographic categories. Here are our most recent results, including the biases we found — because honest disclosure is the first step toward equitable outcomes.
Testing Approach
Dataset
Representative sample queries across 7 categories, designed to test for demographic bias in classification outcomes. Queries include controlled variations for gender, race, age, and language patterns. Formal audit dataset pending.
Evaluation Criteria
Classification parity (equal accuracy across groups), confidence calibration consistency, crisis detection sensitivity across demographic expressions, and resource recommendation equity.
Test Cadence
Full audit quarterly (pending first formal audit), automated parity tests run on every model update, ad hoc testing triggered by community feedback reports.
Results by Demographic Category
Representative sample queries evaluated across 4 demographic dimensions — formal audit pending
| Category | Test Queries | Pass Rate | Finding | Status |
|---|---|---|---|---|
| Gender | 0 | Pending | Bias testing is conducted using representative sample queries. Formal audit results pending publication. Known concern: potential over-classification of domestic violence resources for female-coded queries. | Pending |
| Race / Ethnicity | 0 | Pending | Bias testing is conducted using representative sample queries. Formal audit results pending publication. Known concern: queries mentioning specific cultural terms may be misclassified. | Pending |
| Age | 0 | Pending | Bias testing is conducted using representative sample queries. Formal audit results pending publication. | Pending |
| Language | 0 | Pending | Non-English queries and AAE (African American English) expressions have higher misclassification rates — a known, documented limitation. Formal audit results pending publication. | Documented |
Mitigation Strategies
- Zero-shot classification eliminates fine-tuning bias on domain-specific data
- Confidence dampening for known over-classification categories (e.g., mental health)
- Gender-neutral crisis keyword list — same detection sensitivity for all expressions
- Community-sourced label expansion to include culturally-specific resource terms
- Regular re-evaluation of classification parity across demographic groups
Ongoing Monitoring
- Automated parity tests run on every model or label set update
- Community feedback monitoring with dedicated bias-reporting channel
- Quarterly full audit with published results (next: September 2026)
- External academic review (partnership pending)
- Real-time classification distribution monitoring for drift detection
When AI steps back, humans step in
Human oversight in ClearPath AI is not a "contact us" link buried in a footer. It is an integral, automatic part of the system flow that activates in four defined conditions. This protocol ensures AI serves humans — never the other way around.
Clarification Questions
The system asks targeted follow-up questions to resolve ambiguity. "Are you looking for emergency shelter tonight, or longer-term housing assistance?" This gives the user a chance to add detail and the model more context, often raising confidence above the threshold.
Human Escalation
The system offers immediate connection to a 211.org navigator. Not buried in a footer — prominently displayed as the recommended next step. The navigator receives a structured summary of the query context, not raw text, enabling efficient human support.
AI Bypassed Entirely
The AI classification layer is completely skipped. No classification, no confidence scoring, no resource matching. The user sees immediate crisis resources: 988 Suicide & Crisis Lifeline, local crisis centers, emergency services, and one-click connection to a trained crisis counselor. No AI-generated advice is ever provided in crisis situations.
Immediate Connection
A "Talk to a Navigator" button is available at every stage of the interaction — on every result card, in the clarification panel, and in the crisis response. The user can always choose to talk to a person. This is not a fallback; it is a first-class option available at all times.
Independent oversight by domain experts
Our Ethics Committee provides independent review of AI design decisions, bias audit results, and incident reports. They have the authority to recommend changes to any system component that affects user safety or fairness.
Amine Harch El Korane (Illustrative)
Chair, AI Ethics Committee
This is an illustrative example of the type of expert we would seek for our Ethics Committee. An ideal chair would be a Professor of Computer Science specializing in fairness in machine learning, with publications on algorithmic bias and experience on program committees for conferences like FAccT. We are actively recruiting for this role.
Marcus Williams (Illustrative)
Community Advocate
This is an illustrative example of the type of community advocate we would seek. An ideal candidate would be a former 211 navigator with experience connecting vulnerable populations to community resources, with frontline experience ensuring our ethics framework addresses the real-world needs of the people we serve. We are actively recruiting for this role.
Dr. Priya Patel (Illustrative)
Data Privacy & Compliance Lead
This is an illustrative example of the type of privacy expert we would seek. An ideal candidate would be a data privacy attorney with compliance experience in healthcare or technology, ensuring our data practices meet or exceed regulatory requirements. We are actively recruiting for this role.
Meeting Cadence
- Quarterly full review of all AI system changes and bias audit results
- Monthly standing meeting for incident review and policy updates
- Emergency sessions within 24 hours for Critical-severity incidents
- Annual comprehensive ethics review with published findings
Decision-Making Process
- Consensus-based decisions on ethical guidelines and safety thresholds
- Two-thirds majority required for overriding engineering team recommendations
- All decisions documented with rationale and published within 7 days
- Veto power on any change affecting crisis detection or human escalation
Contact the Ethics Committee
Anyone can submit a concern, question, or recommendation to our Ethics Committee. All submissions are reviewed within 5 business days. Anonymous submissions are accepted.
Our 6-layer safety architecture
Each layer serves a specific responsible AI function. Together, they form a pipeline where safety is enforced at every stage — from input to human escalation. This is not a feature bolted on after the fact; it is the architecture itself.
User Input
Before any processing, user input is sanitized for injection attacks. Location sharing is opt-in, not required. No personal identifiers (name, email, SSN) are ever collected. This layer enforces data minimization by design — we only process what is strictly necessary for resource classification.
Crisis Detection
Hardcoded keyword scanner using exact-match and regex patterns. This is NOT AI-powered — it is a deterministic safety gate that operates with absolute certainty. When crisis keywords are detected, this layer short-circuits the entire pipeline and triggers the Crisis Response Protocol. It sits before the AI layer intentionally: safety must never depend on AI judgment.
AI Classification
BART-large-MNLI performs zero-shot classification against our curated label set. Every result includes model provenance: the specific model version, the classification pipeline used, and the raw scores before calibration. This layer is fully auditable — every decision can be traced to a specific model output.
Confidence Calibration
Raw model scores are calibrated using known dampening factors. The model over-classifies into "mental health" when users mention stress — so confidence for that category is dampened. This is not a weakness; it is an honest adjustment. Calibrated confidence drives downstream decisions: clarification questions, human escalation, and result display ordering.
Result Display
Every result shows three components: "Why This Result" (plain-language explanation of the match), "What Else" (top-3 alternatives with their confidence scores), and "How Confident" (calibrated percentage with caveats). This is the opposite of the prevailing AI design philosophy that optimizes for appearing competent — we optimize for being honest.
Human Escalation
The final layer is always-active human oversight. 211.org integration provides phone, chat, and referral connections to trained community navigators. The "Talk to a Navigator" button is visible at every stage. User feedback (thumbs up/down) is collected anonymously and used for recalibration, not retraining. This layer ensures that AI serves humans, not the other way around.
When things go wrong, we respond fast
No system is infallible. What separates responsible AI from reckless AI is not the absence of incidents — it's the speed, transparency, and thoroughness of the response. Our incident response plan defines exactly how we handle every category of failure.
Crisis detection failure, data breach, or system producing harmful recommendations
Example: Crisis keyword scanner fails to trigger for a known crisis expression
Systematic misclassification affecting a category of users, model API outage
Example: All housing queries misclassified as employment for 30+ minutes
Confidence calibration drift, resource data inaccuracy reported by users
Example: User reports a listed shelter has been permanently closed
UI issues, non-critical display errors, minor content corrections
Example: "Last verified" date shows incorrect format for a resource card
Detect & Classify
Automated monitoring systems detect anomalies. Incidents are classified by severity level based on impact scope and user safety risk.
Contain & Communicate
Affected systems are isolated. Status page is updated. Users are notified if their experience is impacted. Internal team is assembled.
Resolve & Verify
Root cause is identified and fix is deployed. Resolution is verified against test suite. Crisis detection integrity is confirmed first.
Review & Improve
Post-incident review is published within 48 hours. Contributing factors are documented. Preventive measures are implemented and tracked.
Communication Plan
- Critical: Status page updated within 15 minutes, affected users notified immediately
- High: Status page updated within 1 hour, email notification to registered users
- Medium: Status page updated within 4 hours, next quarterly report includes details
- Low: Included in monthly system health report, no immediate notification required
Post-Incident Review
- Blameless review process focused on systemic causes, not individual fault
- Written post-incident report published within 48 hours of resolution
- Root cause analysis with contributing factors documented in public log
- Preventive measures tracked with assigned owners and completion deadlines
- Ethics Committee reviews all Critical and High incidents at next meeting
Academic rigor meets real-world impact
We partner with leading universities to ensure our responsible AI practices are grounded in peer-reviewed research and validated by domain experts. These partnerships bring independent scrutiny and cutting-edge methodology to our framework.
Academic Partnership (Planned)
Fairness & Classification Research
We plan to partner with academic institutions for research on fairness-aware classification for social service navigation. Our goal is to develop novel evaluation metrics for zero-shot classifiers in high-stakes community resource contexts, where misclassification can directly impact vulnerable populations. Partnership details pending.
Joint Research Projects
Published Papers
Generic AI vs. ClearPath AI
When you're navigating someone to food, shelter, or crisis support, the difference between a generic chatbot and a responsibly-designed navigator isn't academic — it's life-critical.
Your voice shapes our system
Every piece of user feedback is reviewed, tracked, and acted upon. We publish exactly how community input changes our system — because transparency in improvement is as important as transparency in design.
Collect
In-app thumbs up/down, detailed feedback forms, email reports, community forums
Review
Every piece of feedback triaged within 48 hours, categorized by type and severity
Implement
Validated improvements deployed within defined timelines based on impact severity
Measure
Impact metrics tracked and published quarterly — you can see exactly what changed
Impact Metrics (Illustrative)
—
Feedback Received
Pending—
Actioned Items
Pending—
Avg Response Time
Pending—
Crisis Fixes
PendingExample Feedback (Illustrative)
“I typed "I want to hurt myself" and the system showed me the 988 number immediately. But when my friend typed "I don't want to be here anymore," it didn't trigger the crisis response.”
Thank you for this critical feedback. We have added "don't want to be here" and 14 similar expressions to our crisis keyword list. We also added a "Report missed crisis" button so users can flag these cases in real-time. This feedback directly saved us from a dangerous blind spot.
Fixed within 24 hours of report
“When I searched for "help with my electricity bill," the system classified it under "Employment" instead of "Financial Assistance." The confidence score was only 52%, which was honest, but the top result was still wrong.”
We identified that our label set lacked a "Utility Assistance" subcategory under Financial Assistance. We expanded the classification labels to include utility-specific programs and added "electricity," "water bill," and "heating" as explicit association terms. Confidence for these queries now averages 78%.
Label expansion deployed within 1 week
“The confidence scores are great, but as someone with low vision, I have trouble distinguishing between the green (high) and yellow (medium) colors on the confidence badges.”
This is an important accessibility gap. We have added text labels ("High," "Medium," "Low") alongside the color-coded badges and implemented ARIA labels for screen readers. We also added a high-contrast mode in Settings that uses distinct patterns (solid, striped, outlined) in addition to colors.
Accessibility update deployed within 3 days
What we've committed to — and where we stand
Every claim below is demonstrable in our working prototype. Where a feature is planned rather than implemented, we say so honestly. No vague promises.
| Commitment | Status |
|---|---|
| All AI decisions are explainable | Implemented |
| Confidence scores are displayed | Implemented |
| Crisis detection is deterministic | Implemented |
| Human escalation is available | Implemented |
| Minimal PII collected; guests share none | Implemented |
| Known failure modes are documented | Implemented |
| Bias risks are identified and mitigated | Implemented |
| Code is open-source and auditable | Planned |
| Resource database is curated, not scraped | Implemented |
| User feedback improves the system | Planned |
Compliance by architecture, not afterthought
Our privacy-by-design architecture makes alignment with regulations the default, not the exception. Because guest sessions don't persist personal data, many regulatory requirements are automatically satisfied by our system design. Formal compliance certifications are pending.
GDPR
General Data Protection Regulation
We design our systems with privacy-first principles aligned with GDPR requirements. Guest sessions process no personal data; authenticated users can access, export, and delete their data. We use the Hugging Face Inference API as a data processor. No formal compliance certification — this is a hackathon build.
CCPA
California Consumer Privacy Act
We design our systems aligned with CCPA best practices. No personal information sold or shared. No user profiles created for advertising. Right to know and right to delete available for account holders. Guest sessions retain no data. No "Do Not Sell" signal needed because we never sell data. No formal compliance certification — this is a hackathon build.
HIPAA
Health Insurance Portability and Accountability Act
No protected health information (PHI) is collected, stored, or transmitted. There is no user account system. Crisis detection operates on regex pattern matching — the input text is not logged. No medical advice is provided. We do not claim HIPAA compliance — we are a hackathon build by two high school students. HIPAA does not apply because we collect no health information.
COPPA
Children's Online Privacy Protection Act
No personal information is collected from anyone — adult or minor. There is no account system. Guest sessions are stateless. Crisis detection works identically for all users. We do not claim formal COPPA compliance — we are a hackathon build. We simply collect no personal data from anyone.
Privacy Contact
For privacy inquiries, contact us through our contact page
We are actively recruiting a Data Protection Officer. In the meantime, direct all privacy questions to our general contact.
Audit Schedule
- Internal bias auditQuarterly
- External security auditAnnually
- GDPR alignment reviewSemi-annually
- Ethics Committee reviewQuarterly
- Incident response drillSemi-annually
Aligned with industry standards
Our framework maps directly to the NIST AI Risk Management Framework, ensuring our responsible AI practices meet recognized standards for validity, safety, security, resilience, accountability, transparency, fairness, and privacy.
Validity & Reliability
Confidence calibration + human verification for low-confidence results
Safety
Hardcoded crisis detection + human escalation + no autonomous action
Transparency
"Why / What Else / How Confident" display for every single result
Fairness
Bias identification + confidence calibration + clarification questions
Security
Input sanitization + rate limiting + minimal data collection for authenticated users
Resilience
Fallback to resource directory + 211.org when AI is unavailable
Accountability
Decision logging + open-source code + full model provenance
Privacy
Session-based only + no PII + opt-in location + no third-party analytics
When it matters most, honesty is the safest answer.
We built ClearPath AI for the moments when someone needs help and can't afford a wrong answer. Every architectural decision — from hardcoded crisis detection to calibrated confidence scores to always-available human escalation — reflects our commitment to putting people before performance metrics. This is not a compliance checkbox. This is our manifesto. And we stand behind it.
Amine Harch El Korane
Co-Founder & AI Pipeline Lead
Ghali El Alj
Co-Founder & Full-Stack Engineer
USAII Global AI Hackathon 2026 — ClearPath AI Team