AX Across the Entire Development Lifecycle, from Code Generation

-From AI coding to testing and operations, how should the development process change?- 1. The way we delegate coding to AI has changed

Within just a few months of beginning to use AI seriously in development, I have almost stopped writing code directly myself. At first, I would explain the functionality I needed and ask AI to generate the code. Now, I plan development goals with AI, create a detailed plan, and proceed through implementation and verification.

I also use AGENTS.md to explain the project's structure and rules in advance and make AI work within those rules. Through SKILL.md, I provide only the context needed for each task, allowing AI to focus on it. If AGENTS.md contains common rules that always apply across the entire project, SKILL.md is more focused on selectively providing the knowledge and working methods required for the current task. Multiple Skills can also be combined for a single task.

Depending on the situation, I sometimes divide multiple models by role. I assign complex analysis and design to high-performance models, while delegating code generation that consumes many tokens to relatively inexpensive models. I may then use another model to cross-check for omissions and assess the completeness of the implementation. The reason for using multiple models is to reduce costs while reviewing results from different perspectives. Of course, if money is no object, using only the top-tier model is also a good option.

2. An enormous amount of code has passed the tests—is it safe to relax?

AI can generate much more code in a far shorter time than before. The problem is that as the amount of code increases, it becomes more difficult for people to review every line manually. When you ask AI to implement a feature, it no longer just writes the code. It also creates the necessary test code. The build succeeds. All JUnit tests pass. Even when I ask another AI to review the code, it says there are no particular problems.

So, can we feel safe now? If we think about it a little, not really. AI created the code, and AI also decided which tests were necessary. The fact that the program passed several tests that AI considered necessary does not mean that it is sufficiently safe. Of course, this is not a problem unique to AI-generated code. Even when we created programs ourselves, there was no 100% perfect software. Problems we had not anticipated occurred in production even after sufficient testing and deployment, so we checked the logs, analyzed the cause, and modified the code. We then added tests to prevent the same problem from occurring again.

This process is software engineering that we have been practicing for a long time. The principle does not change simply because AI creates the code. However, because code can now be generated much faster, I think our verification methods should also change to match that speed. In particular, a simple screen error should not be treated at the same level as problems such as the following:

Authorization issues that allow access to another user's data

• Attacks using abnormal input

• The same request being processed multiple times

• Data errors that occur when multiple requests arrive simultaneously

• Problems in which payment or settlement amounts are processed incorrectly

• An execution order that the developer never considered

• If the way we create code is changing to fit the AI era, the way we verify it should change as well.

3. Let's add AI to test automation, too

For example, suppose we have created an API for modifying member information. Typically, we might think of tests like the following. A person might check a few cases and move on, but we can ask AI to generate many more variations of these scenarios.

That does not mean we need to build a huge, separate testing system from scratch. AI coding tools can already read a project's code, execute commands, and check test results and error logs. We can start by having them run the JUnit tests we already use, analyze the failures, and add new test cases. For web interfaces, we can use existing browser automation tools such as Playwright or Selenium, and for security verification, we can use tools such as ZAP. Tools that provide AI-based test generation and modification themselves, such as the recent versions of Playwright, are also emerging.

Ultimately, rather than building new testing tools from scratch, we attach AI before and after tools we already use, such as JUnit, Playwright, Selenium, and ZAP. Just as we already do for code generation, we can have the testing process repeatedly create scenarios → execute them → check the results → add more tests where coverage is insufficient.

Even this alone could significantly reduce the amount of testing that developers previously had to think through and write themselves. However, one limitation remains. No matter how many tests we create, they still operate within the range we considered during development. So where can we find problems that we failed to anticipate?

4. Can we feed data from production back into our tests?

No matter how many tests we run, unexpected problems occur in actual production. This is not a new concept either. We have always checked production logs and monitoring information to find the cause of problems, fixed them, and added tests to prevent the same problems from occurring again.

In the AI era, couldn't we automate this process a little further as well?

In fact, this is not a completely new approach either. It is something we have used for a long time. Existing software engineering does not disappear just because we are in the AI era. Rather, it is about adding AI on top of established methods and gradually automating tasks that people used to perform extensively: analyzing problems, coming up with new test cases, and feeding issues discovered in production back into testing.

5. Do we need a huge solution before we can get started?

I have not yet implemented this entire structure myself. Therefore, there is still much I need to study before I can speak definitively about specific implementation methods. However, many of the technologies we need already exist.

JUnit for Java testing

• Selenium or Playwright for browser test automation

• ZAP for security testing

• n8n as an automation tool for connecting the workflow

• A local LLM when it is difficult to send internal code or production data externally

• For example, in a very simple setup, a local LLM creates test scenarios, and tools such as JUnit, Playwright, Selenium, or ZAP execute them. AI then analyzes the results and logs and adds new tests. If the environment makes it difficult to send internal code or production data outside, this AI can be configured as a local LLM. A tool such as n8n can connect these steps.

For example, an extremely simple diagram might look like this.

There is no need to build a huge system from the beginning. Just as we are doing with code generation today, we can start by assigning a single task to AI and gradually expand the scope of automation if it proves effective.

Conclusion

These days, many companies talk about AX (AI Transformation). But if we say we will transform our customers' work and systems around AI while our own development practices remain exactly the same as before, isn't that a little strange? I believe using AI to create code is only the beginning of this change. If AI has changed the way we create code, then AI should also change the way we test and analyze test results.

Of course, we cannot change everything at once. As we change things one by one, the entire process of creating software will eventually adapt to the AI era. I believe what matters now is not creating perfect code in a single attempt, but building a structure that allows us to discover problems in imperfect software more quickly, verify them more quickly, and feed that experience back into the next development cycle.

When the change that began with code generation extends step by step to verification and operations, won't the development process itself finally enter the AX era?

zacca

zacca