Building an Autonomous Content Engine People Actually Read: Part One

Many people are struggling with how to adopt AI. They see the potential but lack the ability to harness it.

The problem is process. Most people don't have a way of integrating AI into their workflow. They either go full AI from the beginning and find the system falling short or do something entirely manually.

This article isn't an AI generated article, because I wanted to document my process of how I adopted AI in my writing.

How I previously wrote

I used to have a relatively popular Medium channel. Reads for articles were regularly in the hundreds of thousands views. Not enough to earn a living writing, but enough for me to be encouraged to continue writing in the public eye.

It was super easy, I'd pop over to the website and start writing. I'm reminded here of Do Things that Don't Scale. If I had been thinking about how I'm going to grow to thousands of followers, I probably would have never explored what was interesting to me.

A Process Emerges

Medium was great, but I deleted it.

Why?

Because I was building Medium views instead of my own audience. I'd rather have a small contingent of readers who were interested in what I was doing rather than hundreds of thousands of views that belonged to medium.

But I missed the ease of publishing an article and having easy distribution.

I tried substack, but that completely fell flat. I started subscribing to a number of channels only to wake up with an ungodly amount of emails in my inbox.

No thanks.

I was in a bit of a pickle because I realized Medium wasn't only the editor... it was my entire writing process. The end of using Medium forced me to look at how I used the tool:

1. Capture the idea: If I had an idea for an article. I'd write a quick title and drop some notes. Collecting ideas was a separate process from writing.
2. Explore the idea: With the draft in place, I'd try exploring the idea more, sometimes writing, sometimes finishing, sometimes just adding a line or two. The goal was enriching and exploring at this phase.
3. Polish for publishing: If I was happy with the article and thought it was in a publishable state, I'd then edit and get it ready so I was happy putting it out in the public. This is a fine line to walk. On one hand you don't want to be a perfectionist, on the other you don't want to publish crap.
4. Distribute the content: This was super easy with medium because I didn't have to think about distribution, I just leveraged their platform. Maybe occasionally I'd drop a post on LinkedIn.

Using Tech and AI to transform my process.

At any point I could ask ChatGPT to map out and architect my process... or even generate content. But quite frankly I find the idea of just handing over something so personal as my writing as repulsive and lazy.

The only reason, I even started writing was because I thought it was fun. Developing the process itself was rewarding in ways I couldn't describe and I'll be damned if I hand that agency over to Sam Altman or Dario Amodei.

I found myself asking: "Can I write something authentic with AI?"

Knowing that AI is essentially just text approximation, I seriously doubted that at first.

But that's kind of key. If you're just focusing on what AI can do, you're entirely missing the point.

I want to build an engine for me to explore ideas and publish findings I'm proud of.

Planning the System

If you're not already familiar with the rule of three, pause and go read that.

I wanted to set an aim to gradually adopt ai and automation in my process instead of setting some goal that I would write a bunch of slop with AI. The rule of three is how I'm handling gradual adoption. i.e. If you need to do something three times, only then should you automate it. Premature optimization is the root of all evil.

Phase One: Idea Generation

At first I started thinking in the way of building... "I need a cron trigger to start the writing process daily". But that was a trap. I found myself focusing on the tech not what I wanted to do.

The first thing I had to do is ask:

"Where do I get my ideas from?"

Daily life. News. Exploring Interests. Reflecting on interactions with people. There wasn't one single source. So the problem I have here is - "how do I aggregate ideas for writing about things I'm interested in?"

I love to make lists so I dove into this and just started writing all of the places I find interesting things from:

• Favorite Youtube Channels
• Favorite Subreddits
• What's trending on LinkedIn
• What's trending news
• Conversations with people
• Seemingly random thoughts and observations
• Feelings

If I'm using AI to generate ideas from content. Youtube is quite easy because there's an XSS feed, but I obviously I don't have the same API to my brain to and have a webhook trigger everytime I feel something. I have no desire to journal with anything other than pen. I'm also not super enthused about the idea of sharing and talking with an LLM to share my feelings and develop Chatbot psychosis...

But I do think it would be nice to be able to trigger content creation based on me jotting down an idea.

Where to capture ideas?

I became comfortable with Medium as my preferred editor. It was readily available cross device and limited context so I could easily achieve focused writing. That and having unsplash at arms reach gave me the ability to easily pepper my articles with images.

So then I decided to land on Ghost CMS. I could easily self-host, have a familiar editor, and write drafts like I previously did.

The gathering point and automation

Okay so now I have a way of capturing ideas from all channels including my brain. But at this point, I need a way of the idea triggering an automated exploration.

That was easy for youtube, like I said, I could do something like this:

Youtube -> Feed API -> Extract Topic -> Idea Exploration Function

But needed the same for my Brain.

Ideas / Feelings -> Ghost Draft -> Trigger Webhook -> Idea Exploration Function

Phase Two: Idea Exploration

Okay so at this point, I know that once I have an idea I jot it down I can reliably trigger an automation.

Now I need to map out that idea exploration.

This is where software engineering experience comes in handy. I basically want to write a function with the following

Input: Topic/Idea

Output: Draft Article

function exploreIdea(idea) {
// do stuff
return draft;
};

Thus begins the process of decomposition on how I explore ideas.

This is precisely is where the value lies in the coming years... having novel approaches to exploring ideas and capturing that heuristic as repeatable and scalable code.

I could write an entire post about this alone, and probably will, but to keep things simple - here is just an workflow of what I do:

With the context and ideas, write a prompt to create a draft article.

I then feed that prompt to my preferred writing model. So the process looks something like this.

The last thing there is "get feedback". I need to be able to add additional context and steer the article in the direction that is interesting to me.

Not exactly sure how to do this at this point, but again this is where practical software engineering comes in handy. I want to get something live, not get lost in the weeds. So I'm going to add the feedback piece to the backlog and just focus on completing a POC right now.

Phase Three: Editing and Publishing

This is probably the easiest piece. Literally just need to run a workflow that edit and revises the piece.

I don't really even see a point to having an AI agent in this flow at the moment. I want a workflow not a semi autonomous system.

Phase Four: Distribution

Before I really dive into distribution, I really just want to start testing the previous phases. I have high confidence based on previous experience that it will be easy for me to connect to content api's so mapping this at this point isn't really worth it. It's better to test the preceding steps to see if there is something worth publishing.

Gluing it All together

I love n8n as a quick POC here... easy for webhooks and calling more custom things down the line if needed.

Rather than try to make anything complicated, I'm literally just going to start with a single workflow and can revise as needed.

To be continued...