docs: add custom non-LLM feedback functions cookbook notebook

[nuthalapativarun]

Description

Creates examples/cookbooks/custom_programmatic_feedback.ipynb — a cookbook showing how to write non-LLM feedback functions. Closes #2416.

The existing custom feedback function docs only showed LLM-based custom functions. Many users want deterministic, zero-cost programmatic checks.

The notebook demonstrates 4 non-LLM feedback functions:

1. JSON schema check — validates response is valid JSON with required fields and correct types (answer: str, confidence: float 0–1, sources: list)
2. Regex match — verifies the answer field meets a minimum length via regex
3. Response length score — scores responses on a continuous scale based on character count being within a target range [50, 500]; scales linearly outside the range
4. PII detection — scans for email addresses, US phone numbers, and SSNs via regex patterns; returns 1.0 (clean) or 0.0 (PII found)

Each function includes:

• Google-style docstring with Args/Returns
• Unit assertions as quick sanity checks
• Integration with Feedback via Select.RecordOutput

The notebook ends with a full run against 4 test queries, leaderboard view, and dashboard launch.

Type of change

• This change requires a documentation update

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[sfc-gh-jreini]

Line #8.    def json_schema_check(response: str) -> float:

instead of writing a custom metric, use the new schema validator added in #2427

Reply via ReviewNB

[sfc-gh-jreini]

Line #11.    def no_pii(response: str) -> float:

use out of the box pii detection here instead via huggingface provider:

trulens/examples/expositional/use_cases/pii_detection.ipynb

Line 4 in e0c9573

"attachments": {},

Reply via ReviewNB

[sfc-gh-jreini]

Line #4.        Feedback(json_schema_check, name="JSON Schema Check")

use Metric api & selectors instead of .on()

you can read more here about the change: https://www.trulens.org/blog/2026/04/14/trulens-27-unified-metric-api-mlflow-integration-and-more/#unified-metric-api

Reply via ReviewNB