Get Better at Using Prompts With Deliberate Practice: One technical writer's little experiment — guest post by Diana Cheung

by Diana Cheung on Apr 23, 2024
categories: ai

• Introduction
• Deliberate Practice
• The Experiment
  • The Technical Writing Assignment
  • Brainstorming
  • Selecting Tasks
    • Task: “Use informal but professional language.”
    • Task: “Puns can be difficult to tackle for our non-native English audience.”
    • Task: “Be consistent in heading levels.”
  • Key Insights
• Next Steps
• About Diana Cheung

Introduction

Have you dabbled in prompting, but haven’t achieved the 2x technical writing output as Tom? Well, me either, but here’s what I am going to do about it.

For the past four months, I have taken a more active interest in learning prompting, generally about LLMs and prompt structures and specifically on how to apply it to technical writing. I’m following AI blogs, taking free online courses, and watching YouTube videos promising the perfect prompt formula.

Despite having learned and played around with various prompts, I haven’t used them consistently in my actual technical writing workflows. Prompting hasn’t been fully integrated into my process. It still doesn’t feel second nature to me yet, so sometimes I forget to use it.

So how can I get better at incorporating the use of prompting in my technical writing processes and projects?

Deliberate Practice

Taking a step back though, learning something new is challenging. It takes practice and time to make it your own. Just like when learning to code, reading and copying someone else’s code won’t make you a better programmer. You need to understand the fundamentals, practice, and make mistakes then learn from them. It’s probably the same with prompting.

The concept of deliberate practice is defined as purposeful and systematic practice, not just mindless repetitions. Feedback is one main difference between deliberate practice and simple repetition. One way of receiving feedback is via measurement, such as tracking and evaluating your progress.

Deliberate practice can sound overly rigid and theoretical. There’s also no standardized definition of what deliberate practice looks like for using prompting in technical writing. I interpret it as an approach that includes intention, experimentation, and iteration.

So for my next technical writing task or project, I’m planning to be more deliberate about using prompts in my workflow. I will brainstorm where and how to apply prompting in my process. I will see what works and doesn’t work for me. Although I don’t have quantitative measures, I can still gauge with qualitative observations.

The Experiment

The Technical Writing Assignment

I picked up a GitHub issue for an open-source API documentation tool.

It’s a suitable issue to leverage AI and prompting as there are clearly defined requirements that need to be applied to multiple pages.

Brainstorming

I brainstormed on how to use AI and prompting for each task requirement described in the issue.

• “Meaningful introduction, explanation, and use case for each command.”
  • Thoughts: “Meaningful” seems subjective. Provide the source code of a command and ask AI to generate an introduction, explanation, and use case that are concise and helpful. Perform this task for one command manually first and then use it as an example in a prompt template.
• “Show examples of each task or operation rather than each parameter or value.”
  • Thoughts: Find good examples (task/operation) and bad examples (parameter/value) to provide to AI. Then ask AI to detect instances of bad examples and suggest corrections.
• “Use formatting wisely, consider which segments need to be copyable (use a code block), which do not (e.g. command output), and try to keep the use of admonitions to 2-3 per page. Be extra careful with tabs: a user should be able to pick a tab and read continuously through the document. If we make people click on each tab in turn, the tabs should be removed.”
  • “Use formatting wisely.”
    • “Consider which segments need to be copyable (use a code block), which do not (e.g. command output).”
      • Thoughts: Not sure what replacement Markdown block to use that would still provide syntax formatting and highlighting.
  • “Try to keep the use of admonitions to 2-3 per page.”
    • Thoughts: Ask AI to detect the number of admonitions per page and flag if it exceeds three.
  • “Be extra careful with tabs: a user should be able to pick a tab and read continuously through the document. If we make people click on each tab in turn, the tabs should be removed.”
    • Thoughts: Not sure how to instruct AI on this requirement.
• “Be consistent in heading levels and using active voice for task-based headings throughout.”
  • “Be consistent in heading levels.”
    • Thoughts: Ask AI to list out the headings and levels, then check for inconsistencies.
  • “Use active voice for task-based headings throughout.”
    • Thoughts: Ask AI to check headings (excluding Introduction, Usage, Options, Examples) for active voice and suggest corrections.
• “Use informal but professional language, sometimes the puns can be difficult to tackle for our non-native English audience.”
  • “Use informal but professional language.”
    • Thoughts: Ask AI to analyze the tone of existing documentation and offer suggestions for improvement.
  • “Puns can be difficult to tackle for our non-native English audience.”
    • Thoughts: Ask AI to detect puns/idioms in existing documentation and offer replacement suggestions.
• “Every code snippet must be preceded and followed by explanatory text.”
  • Thoughts: Perhaps ask AI to detect code snippets and any preceding and following text? If text is not present, ask AI to offer suggestions based on the content of the code snippet.

Selecting Tasks

For my assignment, I selected a couple of tasks to leverage AI and prompting. It was too overwhelming to use prompting for all the tasks when I’m still in the early learning stage. It would take too much time and mental energy. Moreover, I didn’t have any ideas on how to instruct AI on certain requirements, such as “Be extra careful with tabs.”

For the selected tasks, I followed a simple process:

1. Create prompts
2. Run prompts
3. Capture results
4. Note observations

Task: “Use informal but professional language.”

I crafted a prompt to detect the tone of the existing documentation and offer suggestions for sections not matching the desired tone of informal but professional. I fed the AI the preview command file.

I realized that describing tone is subjective and open to interpretation by both humans and AI. Claude analyzed the tone of the existing documentation to be formal and technical. The suggestion “Hey, just so you know …” seemed too informal to me. I edited the desired tone for the prompt to be formal and technical, but approachable. I also added specifics for the suggestions.

The suggestions from the revised tone prompt better fit my expectations.

Task: “Puns can be difficult to tackle for our non-native English audience.”

I built upon the revised tone prompt to look for puns, idioms, and other literary constructions in the existing documentation. I used the preview command file.

For more comprehensive testing of the prompt, I should create a test file containing puns, idioms, and other literary constructs. I made a note to do so later.

Task: “Be consistent in heading levels.”

I created a prompt to detect the consistency in heading levels. For this prompt, I tested it with the join command file.

I had to manually correct the AI that the Usage Level 3 heading should be a Level 2.

While the AI gave some helpful suggestions, the output table didn’t list all the entries or only entries requiring edits. The prompt isn’t robust enough. I made a note to further improve it if I come across other relevant prompt examples.

Key Insights

After performing the little learning experiment, I made the following reflections:

• Resist the urge to go back to my usual ways. In the learning stage, AI prompting feels unwieldy and slow, so there’s a big temptation to revert back to the old ways of doing things. The thought “It’s faster to do it manually myself” comes up. I need to put in the effort upfront and accept the discomfort and slower pace. Moreover, it takes additional time to note down observations and action items.
• I will add new steps of AI prompt brainstorming and experimentation to my technical writing process. When starting out, just picking a couple of tasks to leverage AI and prompting is more realistic and less overwhelming. Deliberate practice is an iterative process.
• The number of prompts can grow quickly. I need to find an easily accessible place to store and organize my prompts. Currently, I have a basic solution of storing the prompts in a Google doc with useful headings and descriptions. However, I will eventually need a more advanced tool, such as a prompt registry.

Next Steps

So devise your own experiment to deliberately get better at prompting and infusing it into your technical writing process. Be open to updating your process, by adding steps for exploring how to use AI and prompting. Also, be ready to add in new tooling as you progress. Please share what experiment you plan to try.

About Diana Cheung

Diana Cheung is an AI enthusiast and technical writer focused on the developer experience. She is an avid learner and has a soft spot for tea and meows. Follow her on Medium and Linkedin