Filed under:

Perrin Lindelauf

Draftsmith: An Editor’s Review

An oversized sheet of paper with edited text flows over and behind a laptop screen. A robot holds a magnifying glass up to the text.
An oversized sheet of paper with edited text flows over and behind a laptop screen. A robot holds a magnifying glass up to the text.
Copyright: normaals

Intelligent Editing’s PerfectIt has long been the consistency software standard for editors, so when the rapid AI advances of the last year began flipping industries on its head, I grew curious about how the company would keep pace. 

Its response is Draftsmith, a new AI writing refiner that promises to bridge the gap between lacklustre AI-generated text and the final, human-approved draft. Built on Microsoft’s Azure OpenAI Service, Draftsmith, like many text-wrangling services that have popped up over the past year, is based on OpenAI’s ChatGPT. The difference lies in the secret sauce of extra training that the AI model receives to limit and focus its area of expertise. For editors, Draftsmith’s merits become clearest when compared to ChatGPT’s standard setting (and the way Draftsmith avoids some of ChatGPT’s pitfalls). 

Function and privacy

Foremost, Draftsmith is a Microsoft Word plugin. This means that it can edit manuscripts directly — it isn’t necessary to give a prompt to ChatGPT and then paste the results in. 

Second, it works with Track Changes, inserting edits with your username and making the minimal number of changes necessary. In contrast, you can paste an error-filled paragraph into ChatGPT for grammar and typo fixes, but it merely provides the corrected text without highlighting or tracking each edit. This forces the user to search for each change and manually type them into the manuscript they are editing. 

Third, by getting at OpenAI through Azure, Draftsmith keeps your text encrypted and private (whereas all text entered into ChatGPT is used to further train and refine the AI model). 

Finally, since Draftsmith doesn’t offer text generation, it doesn’t fall into the pitfall of hallucination — making up nonsense — that ChatGPT can sometimes suffer from. Thus, before we even look at specific features, it is clear that Draftsmith gets around many concerns that editors have about AI services in their work. 

Unfortunately, Draftsmith isn’t able to completely avoid the intellectual property and copyright criticisms that face all AI tools based on OpenAI: its Large Language Model was built by scraping the internet over the past eight years, studying Reddit posts and Wikipedia, and processing internet-based book collections. Whether OpenAI has committed wholesale infringement or fair use remains to be seen.

Draftsmith’s toolset

Draftsmith offers a suite of tools to aid in the revision process for writers. These have varying degrees of utility for editors. Choosing a tool opens a sidebar with three text-refining tools and a text window to display the proposed edit. 

For this review, I have focused on the tools I believe will be the most useful for editors: Editing Helper, Readability Tuner, Plain English Converter and Word Count Trimmer.

Screenshot of a toolbar for the Draftsmith plugin for Microsoft Word. A series of purple icons appear: Launch Draftsmith, Fluency Enhancer, Dictation Fixer, Engagement Tuner, Word Count Trimmer, Empathy Tuner, Readability Tuner, Plain English Converter, and Editing Helper.

Copy editing with Draftsmith

[Editor’s note: This passage has been updated to reflect a clarification to the scope of how much text can be edited at once.]

The Editing Helper offers Redraft (a copy edit pass with restructuring for clarity), Polish (a copy edit without sentence restructuring) and the self-explanatory Remove Typos. 

Unlike PerfectIt (or competitor Grammarly), which will chew on long manuscripts, Draftsmith is primarily aimed at sentence-by-sentence editing. This focus is usually fine for Redraft, as most editors will want to consider each sentence revision individually, but the lack of awareness of preceding and following sentences may create stylistic problems within paragraphs if the tone abruptly changes. It is possible to edit up to 400 words at a time, which works best for the Word Trimmer (below) or Polish tools. I did find myself wishing that Remove Typos could handle a whole document like a smarter spell check.

I ran the famous first line from Jane Austen’s Pride and Prejudice through both Redraft and Polish.

Screenshot of the Editing Helper: Redraft window in Draftsmith. The first line of Pride and Prejudice (used a sample text) has been changed from "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife" to "It is universally acknowledged that a man who is single and wealthy must be in search of a wife."
Screenshot of the Editing Helper: Polish window in Draftsmith. The first line of Pride and Prejudice (used a sample text) has been changed from "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife" to "It is a truth universally acknowledged that a single man in possession of a good fortune must be in want of a wife" (i.e. the archaic commas have been removed).

As you can see, Redraft prefers a more straightforward structure, whereas Polish retains Austen’s structure but dispenses with the archaic commas. In both cases, Draftsmith does a good job of not introducing errors.

Adjusting readability

Next in line for editors would be the Readability Tuner and Plain English Converter. The first offers buttons to simplify text to target one of two Flesch-Kincaid reading levels (Eleven-Year-Old Reading or High School Reading). The Simplify button rewrites with shorter sentences, simpler grammar and more common vocabulary. 

The Plain English Converter offers the Simplify button once again — indeed, many of the tools share features. (Initially, the reuse of tools seemed redundant, but Intelligent Editing is trying to offer the most useful tools through the lens of a particular task.) There is also a Plain English tool to avoid jargon, technical terms and wordiness. The College Reading tool, on the other hand, will either make text simpler or more complex to achieve this Flesch-Kincaid level. 

To test these features, I ran the opening line of A Hitchhiker’s Guide to the Galaxy (Douglas Adams) through.

Screenshot of the Readability Tuner: Plain English window in Draftsmith. The first line of The Hitchhiker's Guide to the Galaxy has been revised. Original: "For out in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy lies a small, unregarded yellow sun." Revised: "There is a small, unimportant yellow sun located in the unfashionable end of the western spiral arm of the galaxy."
Screenshot of the Readability Tuner: Simplify window in Draftsmith. The first line of The Hitchhiker's Guide to the Galaxy has been revised. Original: "For out in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy lies a small, unregarded yellow sun." Revised: "In the western spiral arm of the Galaxy lies a small yellow sun."
Screenshot of the Plain English: College Reading window in Draftsmith. The first line of The Hitchhiker's Guide to the Galaxy has been revised. Original: "For out in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy lies a small, unregarded yellow sun." Revised: "The small, yellow sun lies in the unfashionable end of the western spiral arm of the Galaxy, far from the known regions."
Screenshot of the Plain English: Eleven-Year-Old Reading window in Draftsmith. The first line of The Hitchhiker's Guide to the Galaxy has been revised. Original: "For out in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy lies a small, unregarded yellow sun." Revised: "In the far reaches of the Galaxy's western spiral arm, there is a small, unimportant yellow sun."

Plain English is basically straightforward, although I wonder whether “unpopular” would be plainer than “unfashionable.” Simplify does pare the sentence down, but it loses the sense that a story is being told. College Reading is interesting: it keeps the prepositional phrase at the end but dispenses with the five-dollar words. I guess these hypothetical college kids aren’t English majors. Another surprise: the Eleven-Year-Old Reading tool maintains the inverted structure but swaps in “there is” for “lies.” 

Reducing word count

The Word Count Trimmer tool is useful for editors trying to meet word count targets. It aims to cut extraneous words without impacting meaning. However, the Simplify tool can save even more words through its dramatic revisions. Redraft, appearing again from the Editing Helper, lets editors consider different structures that may be somewhat shorter. This time, I tested the lengthy (119-word) intro sentence to A Tale of Two Cities (Charles Dickens). 

Screenshot of the Word Count Trimmer: Reduce Word Count window in Draftsmith. The first line of A Tale of Two Cities has been revised. Original: "It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair; we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way--in short, the period was so far like the present period that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only." Revised: "The period was both the best and worst of times, the age of wisdom and foolishness, the epoch of belief and incredulity, the season of Light and Darkness, the spring of hope and winter of despair. We had everything and nothing before us, all going direct to Heaven or the other way. The period was so like the present that some insisted on its superlative degree of comparison."
Screenshot of the Word Count Trimmer: Redraft window in Draftsmith. The first line of A Tale of Two Cities has been revised. Original: "It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair; we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way--in short, the period was so far like the present period that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only." Revised: "The period was one of the best and worst of times, characterized by wisdom and foolishness, belief and incredulity, light and darkness, hope and despair. Everything and nothing were before us, and we were all going direct to Heaven or the other way. Some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only."
Screenshot of the Word Count Trimmer: Redraft window in Draftsmith. The first line of A Tale of Two Cities has been revised. Original: "It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair; we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way--in short, the period was so far like the present period that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only." Revised: "The period was one of the best and worst of times, characterized by wisdom and foolishness, belief and incredulity, light and darkness, hope and despair. Everything and nothing were before us, and we were all going direct to Heaven or the other way. Some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only."

The Reduce Word Count tool helpfully says that 50 words were cut, something I wish were included in the other two tools. Doing the math myself, I found that Simplify cut out 65 words, and Redraft cut 54 words — but when I clicked back to the Redraft results to count, Draftsmith generated a totally new revision of the opening line, this time cutting 51 words. 

I was surprised to lose my initial results so easily; perhaps the developers will put in a “pin” feature to save good results temporarily while you play with other tools.  

Final thoughts on Draftsmith

Draftsmith is an intriguing toolbox from a trusted app developer in the editing community. By working within Word, with Track Changes and with privacy in mind, it brings AI text manipulation to the editing field and answers many qualms that my colleagues have expressed. 

This is certainly a 1.0 release, but as the app develops, I hope it is eventually able to process more text at once. I see myself using Redraft and Simplify on tangled, jargony sentences to get editing options when I would otherwise spend several minutes pulling out what’s left of my hair. 

___

The Editors’ Weekly is the official blog of Editors Canada. Contact us.

4 Comments on “Draftsmith: An Editor’s Review”

  • Gael Spivak

    says:

    Thanks for the review. It’s generous of you to write up and share your experience with people.

    About the readability aspect of this program. I avoid using readability formulas now. Plain language experts recommend not using them. Here are some of those reasons:
    • They don’t measure comprehension.
    • They were not designed for the type of texts we usually edit.
    • They undermine many proven plain language techniques.

    While someone might argue that hey, at least they catch some mistakes, what they do catch is pretty easy to learn how to spot and fix. And research shows that people who use these formulas will write to get a lower score (instead of writing to meet audience needs). So they really aren’t worth using.

    For me, including readability formulas in any program indicates not being up to date on current thinking in the field of plain language.

    Reply

  • The problem with tools like this is that, unless the human editor checks every word that has been changed or added, big mistakes are almost a certainty. It also makes a person lazy, thinking they can do twice the work. Sure, experiment with it, play with it, but do not incorporate what the program has done into the article or manuscript.

    Reply

  • Andrew Park

    says:

    I am admittedly biased against AI for writing (or editing) original content. So, you could view my opinion as less than objective.

    However, the main impression I get from the various edits done by Draftsmith is that the voice of the original writer has been leached out of the prose and cadence has been lost. That seems to be especially the case in the Dickens and Adams passages. So, the exercise, in its favor, did highlight for me the sheer poetic beauty of the writing turned out by these masters.

    Reply

  • Are there any discounts available for the software for Editors Canada members?

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

To top