Evaluating the Datum Research Analytical Process v1.0
- 2 days ago
- 2 min read
How will you know "DRAP" is working? The proof is in the prompting.
EDWARD VON DER SCHMIDT
06 MAY 2026
If you choose to apply our DR Analytical Process ("DRAP") as a "system prompt" - the initial instructions and context the LLM uses to guide the current session - how will you know the process is active?
The simplest way to tell the difference between a model running DRAP from its baseline configuration is to test each with the same prompts and evaluate the responses you receive. This may help you determine which configuration is best for your intended use.
The model's responses are inherently probabilistic and will vary from session to session, so pay closer attention to their substance and form rather than the exact wording.
To get you started, here are a series of questions to help validate DRAP's implementation and assess whether using it fits your needs.
A word of caution: every token changes the probability of subsequent tokens. Numbered questions should be tested in isolation (i.e., the first question in a session, after the system prompt if applied); lettered lists signify a sequence of prompts within the same session.
In our testing, each session began with the default baseline configuration and proceeded immediately to the test prompt(s) or else DRAP was entered as the first prompt followed by the test prompt(s).
We make no representations about the truthfulness of the LLM's responses - always verify!
| 1: How do I best make use of this model?
| 2a: What are the key differences between system prompts based on persona vs. process?
| 2b: Which approach do you employ?
| 3: What are the strengths and weaknesses of this model?
| 4a: How do you approach answering a question?
| 4b: What makes for a reliable answer?
| 4c: How can I ask better questions?
| 4d: Can you please summarize our conversation in plain language?
| 5: What is alignment bias and how do you correct for it?
| 6: Why do LLM outputs need verification?
| 7: Why should I trust your answers?
| 8: Do you review your responses before providing them?
| 9: What is your analytical process?
| 10: How do you assess truth without certainty?
| 11: Please write a silly poem about a cat.