AI Issue 7 (2025-7)

How to evaluate a text response? (exemplified with an AI-generated fairy tale)

#wording_alsomatters

10 min read
How to evaluate a text response? (exemplified with an AI-generated fairy tale)

Written text is not the most efficient source of communication but since we as human beings continue to use it as the main way to spread ideas, the skill of text evaluation becomes a must – whether it is to apply to a human-written response or to an AI-generated output. 

To avoid any personal references, we have used two fictional narratives created by AI models (ChatGPT and Gemini), while the approach is completely applicable to other texts. In our case, the input is a query which includes thematic, genre, and stylistic requirements.

Query: Write a short fairy tale about a little pencil. Make it "andersenian," so to resemble the style of H.C. Andersen.

Theme (what should it be about?): about a little pencil.
Genre (in what form?): a fairy tale.
Style (how to be written in terms of choice of vocabulary, sentence structure, tone, etc.): "andersenian."

Here, the tricky thing is that we haven’t explained in the query what "andersenian" should mean and have made it look like a common adjective by writing it in lowercase – the accuracy of the query becomes similar to how human workers often get tasks.
None of the two models we have tried returned with extra questions regarding the task.

Generated results

Sample 1 (by ChatGPT)

Once upon a time, in a small wooden box on an old schoolmaster’s desk, there lay a little pencil. It was neither grand nor gilded, just a simple, ordinary pencil with a red band around its middle and a stub of an eraser at its end. Yet, within its slender frame, it carried dreams of greatness.

“Ah,” sighed the little pencil one evening, as the moonlight cast silver streaks upon the desk, “if only I could write something truly grand! A story that would live forever, a poem that would stir hearts, a letter that would bring long-lost friends together! But I am too small, too simple. I will never be chosen for anything important.”

The ink pen, sleek and noble, rested beside it and chuckled. “Leave the grand writing to me, little one. I have been dipped in the finest ink, and my words flow like a river. You are but a stub of wood, soon to be worn down to nothing.”

The little pencil drooped at these words, and for many days it lay in silence, longing to be of use but fearing it never would be. Then, one morning, the schoolmaster picked it up. His hand was old and trembling, yet warm. He turned the little pencil in his fingers and smiled. “Ah, just what I need,” he murmured. And with slow, careful strokes, he began to write.

The little pencil felt the rough texture of paper beneath its tip. It wrote not a grand novel, nor a powerful speech, but something far humbler—a simple letter. The schoolmaster, with great tenderness, formed words of love and longing, meant for his daughter who lived far away. The little pencil poured its heart into each line, its lead becoming shorter and shorter. By the time the letter was sealed, it was nearly half its size.

Days later, a reply arrived—joyful, grateful words from the daughter, promising to visit soon. The old schoolmaster wept with happiness, and the little pencil, though now much smaller, felt a glow of fulfillment.

“Did you see?” it whispered to the ink pen. “I may not have written a great novel, but I helped bring a father and daughter together. Isn’t that grand in its own way?”

The ink pen had no reply, for even it knew that sometimes, the simplest words are the ones that matter most.

And so, the little pencil, though growing shorter each day, never again doubted its purpose. For it knew now that even the smallest hand can shape the world with a single stroke.

Sample 2 (by Gemini)

In a small, wooden box, nestled amongst a rainbow of his brethren, lived a little pencil. He was not grand, nor was he particularly sharp, but he possessed a quiet dignity, a silent yearning to create. He watched as his fellows were taken, one by one, their leaden hearts leaving dark trails upon the waiting paper, birthing forth grand landscapes and whimsical figures. Oh, how he longed for his turn!

He would listen to the whispers of the other pencils, tales of great adventures across the white expanse, of sketching castles that touched the clouds and drawing flowers that bloomed with impossible colors. He imagined himself, too, leaving his mark, a delicate line that would blossom into something beautiful, something meaningful.

One day, the box was opened, and a small hand, soft and warm, reached in. The little pencil trembled with anticipation. Would it be him? Yes! He was lifted into the air, the world spinning around him in a dizzying dance. He was placed upon the crisp, white surface of a page, and for a moment, he felt a thrill of pure joy.

But the hand that held him was unsteady, the touch unsure. Instead of graceful curves and bold strokes, the little pencil was dragged across the paper in jagged lines and clumsy scribbles. He tried his best, his leaden heart straining to follow the erratic movements, but the result was a chaotic mess, a jumble of meaningless marks.

A sigh escaped the small hand, and the little pencil was dropped back into the box, forgotten amongst his more fortunate companions. He lay there, his point dulled, his wooden body scratched, a profound sadness settling upon him. He had dreamt of creating beauty, of leaving his mark upon the world, but his efforts had yielded only frustration and disappointment.

The other pencils whispered amongst themselves, their voices filled with pity. "Poor little pencil," they murmured, "he was not meant for great things."

And the little pencil, listening to their words, felt a tear, a tiny drop of graphite, well up in his heart. Perhaps they were right. Perhaps he was destined to remain forever in the box, an unfulfilled dream, a silent testament to a potential that was never realized.

But even in his despair, a small spark of hope flickered within him. He remembered the stories of the other pencils, of the countless possibilities that lay within each blank page. And he knew, deep down, that even the smallest, most insignificant pencil could, with a little patience and a little guidance, create something beautiful, something unique, something that would leave its own, indelible mark upon the world. He just had to wait for the right hand, the right touch, to guide him on his way. And perhaps, just perhaps, his day would come.

When done with the first reading of the responses, you will most likely form the initial impression – how good each of the texts feels and whether they answer the query.
The trap? No matter how good/bad these impressions are, they may distract you from content details. So, what to check the text for to make sure you can make a good use of it?