Iterate: AI-Assisted Planning and Design (with Kiro) - CDX 26.01 [Video]

Don't let AI be a "Yes"-man. Have AI perform the role of senior-level project manager, security analyst, cloud architect, operations engineer, and software engineer and to iterate, improve, and challenge so a solid specification can be formed. (Image generated for author by ChatGPT Image)

Five anthropomorphic 1980s computer monitors with stick figure bodies, each sitting in a line of chairs taking requirements from a stick figure client on the right. Each show thought bubbles about concerns respective to their role (security, developer, etc.)

Creating production-grade infrastructure with AI requires more than throwing a prompt at a language model, it requires discipline, process, and the right tools.

I recently started a project with ChatGPT to iterate over some design ideas and get recommendations. I then brought those ideas into the Kiro IDE and went through four more rounds of refinement before generating the requirements document and kicking off the spec-driven workflow.

My advice:

• Start with a solid foundation and understanding.
• Iterate over your ideas.
• Have your AI assistant perform the role of a senior-level project manager, cloud architect, security analyst, operations engineer, and software engineer to find gaps, challenge your ideas, and provide recommendations.
• Develop a well-disciplined approach to spec-driven development with solid steering documents and a willingness to spend a day or two iterating over your ideas and requirements.

Whether you're building cross-account pipelines or any complex infrastructure, this workflow will help you leverage AI effectively without sacrificing clarity or control.

I use the Kiro AI-Assisted IDE, but these principles translate to any assistant.

Though I have been featured in an AWS Case Study with Kiro, AWS is not a sponsor.

Links mentioned in video

• Atlantis SAM Template Repository featured
• Atlantis Website

Video

Transcript

Introduction

Welcome. I just want to go over a little bit about what I've been working on and how I've been using Kiro to work on creating some templates.

So, I work -- I have a project called Atlantis that I work with and basically it's a platform that is used to help deploy serverless applications. So, it's a platform framework of templates and scripts that basically help you automate your SAM configurations and deployments on AWS.

It covers everything from managing your pipeline to having starter code--to start a project with--to even Kiro steering documents to help get Kiro off the ground so that you're not so worried about the infrastructure you know the backend infrastructure in terms of the CI/CD the testing and, you know, how things come together with security and all that sort of stuff that you don't need to worry about any of that you just basically get started with essentially a shell a housing frame, essentially, so that you can put your own business process and your own logic and what it is that you need in the Lambda function or cloud infrastructure that you need for your application.

Cross-Account Deployment Challenges

So one of the things that I've been working on recently is creating a pipeline that does cross-account deployments. And a lot of times you'll just see a cross-account deployment be I build it, I run it in a test environment, then I take that artifact and I just, you know, approve it.

Well, I didn't really want to do it that way. I wanted to make sure that each environment has its own build that gets rebuilt for each environment because this platform, this framework relies heavily on conditionals. Like, if it's a test environment, it doesn't need to have all the alarms, bells, and whistles that a prod environment does. If it's prod, it should deploy gradually rather than all at once. That way, if you deploy gradually, you can hopefully pick up some errors and automatically roll back before a lot of your users hit those errors. So, it's more of a controlled environment. And we also wanted to make sure that we also can have a beta environment, too.

That way we can have a prod and beta and maybe do some like traffic switching and blue green deployment as as well.

So in the past how I did that was you have a test branch, a main branch you merge your code changes into test and it deploys a pipeline and that's all good. Then you merge your changes into a main and then it deploys there. Well, for cross-account, I wanted to do something a little bit more stable, and I wanted to be able to deploy an artifact, deploy an application, test it out, and then promote it essentially.

So like once that test passes, I'm going to promote it do a manual promote and then that will then copy that commit essentially as a zip file as an artifact to the next account--production account--and then that will automatically pick up and you can manually or not manually approve--maybe it's automatic--so you can either manually approve or automatically approve which you receive from the test account and then deploy that in like a beta and then you can, once that's good, you can then approve the next step.

Using Chat Client for Design

I basically worked with my-- my web chat client, ChatGPT and basically it knows pretty well how I work in terms of I work. I'm security conscious, I work on AWS serverless, platform engineer, software engineer background, DevOps, and developer experience.

So whatever it is that I create needs to be maintainable as well as usable--and reusable even--amongst a wide spectrum of individuals whether they have a lot of experience or just starting out. So it already knows all that. So when I interact with my chat client, it takes all that into account. So then it gives me better-- it's a better conversation because it already knows where I'm starting from and it already kind of knows where it is that I need to go. It also-- I've requested that it challenges me that it doesn't just take my design ideas that it challenges me to adhere to questions-- that that it should ask me questions that a cloud architect would-- a senior cloud architect would ask, a senior security analyst would ask, and a senior software engineer would ask too if they were reviewing my ideas and my design.

Preliminary Design Document

So after a few rounds with that, I basically came up with this preliminary document-- or actually the AI chat came up with this preliminary document in which it just goes over kind of our thought process some of the ideas that we went back and forth and then the ultimate solution that we decided upon.

All this, I keep all my specs and steering documents in the repository of the project that I'm working on. So if you wanted to check these out, it is underneath the specs document 0-0-40. That's the version that I'm working on right now. Pipeline with promotion approval. I'll put that link up there and then you can go visit the repository on GitHub and you can check it out.

But basically the preliminary document is what we received-- or what I received from my my chat client after we were like hashing things you know design ideas. It was probably about a day or two process. I even like discussed it with our platform our cloud architect as well as a cloud engineer you know to get their input and it seemed like it was pretty much on the right path.

And I knew that there were some things that the AI was giving me that I'm like gosh that sounds a little complicated and I don't know how I would actually do that without having to implement a Lambda function and then if you're implementing a Lambda function and then you're implementing this and that all on top it's no longer lightweight pipeline but those are things that you know could be because I didn't want to deploy a Lambda function for each pipeline.

So a companion Lambda function for each pipeline. because that would mean that if you have 10 pipelines in the account then you're going to have 10 Lambda functions you got to manage those in terms of like it I mean it's serverless but yet at the same time you still got to upgrade any packages that you're using any libraries you're using or even the runtime.

You got to update the runtime. So I really didn't want to go that route. But I really want to appear-- I don't know when you tone it just basic a CloudFormation template. I really don't want to go any further than just being able to deploy a CloudFormation and not have a whole bunch of things to manage. So a lot of stuff that is just managed for me.

Kiro Specification Workflow

So, the preliminary design does kind of go into a few things that I just kind of knew wouldn't make it into the final design, but that was okay. so again, this preliminary design was probably, you know, a few questions that I went back and forth with my chat client.

Now, I don't have my chat client be the designer because it doesn't know my code base. It's just basically an outside consultant type thing. And then I bring what we discuss and with my expertise of knowing the code base and stuff like that I bring it to that so I kind of know and I can kind of answer its questions and steer it in the right way.

But then there's some things that I just don't know much about. So then when I bring the output from a chat client I bring it in here as like a preliminary design kind of like ready for more of a discovery phase essentially. So more more planning and I don't turn this right into a requirements document in Kiro. If you've worked with Kiro, you--

I use the spec-driven workflow. I don't-- I don't vibe code. I have a plan. I formulate it. I plan it out. I iterate. I ask questions. And then I come up with a requirements document.

I have videos on this so I'm not going to go into that.

Basically, I gave Kiro this preliminary document. I said, "We're still planning. Go over this. See if there's any gaps, questions, stuff like that. Look at it as if you're--" I always want it to be looking at it as if it's a senior level security, engineer, cloud architect, and, platform engineer, software engineer. I always tell it look at it like this and find any gaps, find any concerns, find any inconsistencies, find anything that-- and I have a whole list of other specs like lightweight CloudFormation based, you know all that.

Oh, and it because we're working within a framework, a well described framework, look at all the other templates in this repository because it's a repository of templates. Look at all those. See how we define modules. See how we create these other templates. What are the patterns that we use? You know, think about that. The parameters, how we use parameters, how we make sure that they are loosely coupled yet can easily come together and receive the parameters and the values and the conditionals that they need to be able to deploy this and be modular and all that sort of stuff. So there are steering documents and I can show the steering documents for that.

So basically what I'm getting at here is I don't just sit down in front of Kiro and start typing. I have a good library of steering documents that I use for this project, for any project. I do know about cloud development. I'm a software engineer. I've been working with AWS for a while now. I'm certified. I know enough. but there are still areas that I don't know much about. I can kind of like tell when we're getting off track.

So I come to it with some experience but also wanting to be challenged and wanting to have better ways of of doing things.

Multiple Rounds of Questions

So, I have all that. So after I tell it to review this I ask it to ask any clarifying questions, come up with any recommendations, and put that into 'QUESTIONS.md'.

So, it did that. It went through, it had a whole bunch of questions I had to answer them. There was like 20 some questions I think, yeah 21 questions, 22 questions that I had to answer. So, I answered them and and some of them were like oh where I had to clarify and I had to say "no let let's go this route" or it gave me like option A, B and C and I said "let's go with B" or sometimes, you know, I could say "let's take A and let's mix a little bit of C in there" and was able to do that and then I said, "okay I've answered all your questions go ahead and review and are there any more questions? If there are ask them in 'QUESTIONS-2.md'" which it did.

So we went through 'QUESTIONS-2' and this time it you know it was like able to clean up quite a bit because there were a lot of loose ends, unanswered things that it just really wanted to know about because, again, the previous chat did not have any of the codebase.

This is actually looking at the codebase, looking at the idea, comparing it to the codebase and instead of just taking the idea and running with it, it's looking at what's actually there and how would this fit in? What are the patterns that are being used? -- which is very important.

That way, you're not just vibe coding and bolting things on. It's actually look, we're looking at a way to integrate my idea into the existing codebase. Which, if you have a strong codebase, great. If you have a crappy codebase, integrating into the current codebase, you-- garbage in, garbage out. But this is a pretty solid code base here. I mean, it's not really code. It's CloudFormation templates, but it's fine.

So basically it asked more questions if fewer questions. This time it was only like nine questions. So I answered those. I again said, "okay here's those. If you have any questions feel free." And sometimes I even changed my mind a little bit. So then it came through and it had more questions. Again, very few there are a few things where I was like, "oh yeah, I didn't think about that. Let's-- let's clarify that," or, I'm like, "you know I kind of changed my mind on that. Let let's do it that way." So then it did that, said if you have any questions go ahead and it came up with 'QUESTIONS-4.md'.

So a fourth set of questions, which is a record for me.

Increased Developer Understanding Through an Involved Process

I've never gone through like several rounds on a chat client and then four rounds of questions with Kiro.

I will say that after you go through four rounds of questions with Kiro, after like having a pretty solid starting document, or starting plan, starting idea, you really get to know-- it really helps you understand what it's building.

That's sometimes a problem with AI-assisted development in software engineering is that you have the plan, you give it the plan, but you don't really know how it executed. Well, I mean, this just dug into every single little thing. And I could probably, after going through these four rounds of questions, hand-code and come up with nearly exactly what AI came up with.

Because, now I know what modules it's going to create. I know how it's going to connect into the parent templates, what parameter, what CloudFormation parameters it's going to use. I know what conditionals it's going to use. I know what security policy it's going to use, how it's going to do the naming convention, how it's going to create the service role that allows the cross-account promotion, how it's going to gate the deployment, you know, so that you need to say yes, test is good, I'm going to approve it, copy it over, and then wait for approval on the receiving end to say yes, I'll accept this incoming artifact and you know, move on. I know that you don't need to do cross-account. You could even do the same process with the approvals, but in the same account.

I mean, I know how it's going to be doing all this just by it asking me questions and walking through with it. It's really nice to kind of like have that AI companion who, who you can just sit down and work with. You're not working around people's schedules or meetings and stuff like that. You're kind of going over that.

It's nice having me in the loop as a human. It's nice being able to like take this, go to another group of humans and say, "Hey, this is what it's coming up with. What do you think? Any insights? Any inputs? Is there anything that we're not considering with the way that this would work in our environment?"

And just knowing that after all these rounds of questions, and without writing any code yet, that we have a pretty solid understanding-- I already have a solid understanding of how I'm just going to go through and read the CloudFormation templates and know every single line that it's going to do.

So that's-- I'll just put that out there in terms of like when you have a good workflow, great steering documents, and you're-- if you're working with a, you know, a chat agent, that it knows who you are and what you do and it has some background knowledge of all that, it's very, very helpful and that that's how I'm able to come up with these ideas, iterate and basically create production ready, ready to go, not have to, like, wrestle with AI developed applications.

We actually have something that works on first deploy and that's been my experience working-- using this workflow, that it takes a couple of days, but then once I actually have it do all the tasks, I have a ready-to-deploy application. I'm not going back.

Model Selection for Different Phases

Then I go-- and then it created the requirements.

I think I only had like two changes in the requirements, you know, after I read through it-- and again, I read through every single requirement. I read through the whole thing and it's good.

When it gets to the design phase, I kind of-- when I have it create from the requirements the design, I get a little lax.

I read through it, but I'm not-- and I should be probably a little bit more attentive, but it's really going off what the requirements did. And after all that stuff, you kind of have fatigue. So maybe you should just like wait for the next day or have somebody else review it. But, it's like I have an idea. I go through this. Yeah, it looks good. And then I just have to create the tasks.

Now, everything that I described from even when I was working with the chat client, though it was ChatGPT, I was using was it I forget if it was Sol or Luna but I was using a higher-end reasoning model. When it comes to creating the questions and the requirements and the design, I'm using Claude Opus.

And I'm not necessarily using the latest always, but I'm using Cloud Opus. You really don't go through that many tokens. This one I only went through like 300 tokens to go through the preliminary reading, the questions one, two, three, and four, and then requirements, design, and tasks. I use Opus for all that.

Now, I can use and I have used, but where you really burn through tokens is executing the tasks.

And this is something that I've kind of read about. I've did a little bit of research on. I even asked ChatGPT and stuff or not just ChatGPT, but I've also, you know, gone to Gemini as well. I was like, you know, "is there something to having like Opus or, the high version of GPT [Sol], do these preliminary deep dives, structural documents, and then when it comes to task execution doing a mid-tier like Sonnet for example," and it said, "Yes".

It said use Opus for your high reasoning, deep architectural stuff whether it's like understanding a codebase or developing your plan, your specs, all this sort of stuff, use Opus for all that. But when it comes to executing the tasks, you've already done all the high-level reasoning. You're just basically taking that and you're just like translating it to a task.

I mean, you you've already got everything outlined. It already knows how it's going to do everything. You've already got design documents. A lot of times you'll even have like code snippets in the design document. So, it pretty much already knows how--yeah, see, here's a code snippet.

So, it already has all that in there. You're just implementing it. And, in reality, it's just following like a recipe. So, you don't need to use Opus for that. You can use like Sonnet. It did not recommend Haiku or you know a lower level one but, when I go to execute the task I can just switch to Sonnet for that.

Now beyond that, that's pretty much my workflow.

Steering Documents and Standards to Ensure Quality and Completeness

Oh! Steering documents so I do have, for each project, I have a group of steering documents that I use across all my projects, or most of my projects depending upon whether whether it's front end or backend Lambda or whatnot.

This particular one, I have a group of steering documents, and I think as I've fine-tuned these very well, that again when I sit down and I start coming up with a plan, spec-driven thing, it knows to look at the codebase, it knows how I version everything, you know there there's a package version, overall package version, but then each template, each script has its own version numbering. It knows all that. It will increase those.

There's standards in terms of naming conventions, you know for it's the difference between Python variables, Node variables, CloudFormation parameters, you know, how I name stuff, even the logical ID name-- that logical resource ID name that's in each of the templates. It knows about all that. It knows how I want my modules to be just the definition as well as a good comment section about what parameters need to be in the parent template, what conditionals need to be in the parent template as well. It knows the structure and all that sort of stuff.

So it basically goes through that. Oh, as well as documentation, because these templates are being used by others. So the documentation for each template needs to be up to date. So it knows that every time it creates a spec-driven a new template or modifies a template that it goes and updates the documentation.

So that way when I basically am done with a workflow when we've created something, one: not only does it deploy, but I don't need to go back and worry about the tests I don't need to go back and worry about the documentation I already know it's already there. And I don't just assume that those are things-- that because we went through the questions and requirements, I know are going to be there because it was in the requirements document.

So it reduces my cognitive load and it ensures that everything that I produce is solid.

Best Practices for AI-Assisted Development

So my best advice is when using Kiro,

Have a workflow and have a well disciplined workflow with solid steering documents, a solid code base.

A solid set of iterations and planning. Don't just rush into things. Take a few days to go through, iterate through it.

Ask other people questions as well as AI. Challenge yourself. Have your AI assistant act like that senior level security engineer, project manager, software engineer, platform engineer, you know, cloud architect, have them perform at the senior level.

Because then you're going to get quality stuff that is not AI slop that will just work.

Closing and Atlantis Framework

So that's been my workflow. That has worked for me.

I definitely recommend exploring a workflow similar to that because I just feel like you just get the best output.

I mean you you're paying for the tokens. You might as well use them effectively and efficiently and without constantly having to iterate and re-fix things. And this is the workflow that I've used that has worked.

So, thanks for watching. go ahead and subscribe and like this video.

If you have any tips or tricks of your own when working with Kiro, or anything that would translate to working with Kiro, feel free.

I do a lot-- I wouldn't say a lot-- but I do videos and I create resources based upon spec-driven development with Kiro and I work on the Atlantis framework which is basically a small-tier platform that allows individuals and small organizations to get into the cloud quickly.

So, have a good day!

[Sound of crackling fire]