Can AI (ChatGPT+Midjourney) do EA work?
Let’s mix ChatGPT with a little Midjourney and see what comes out…
Early in 2023, two platforms related to AI (Artificial Intelligence) and ML (Machine Learning) started to gain traction and trend across the globe; and although these were not net new platforms, they have evolved with comprehensive data-sets and accessible channels of interaction.
Generally, these tools are meant to produce human-like responses, but by sampling data at scale that would be unachievable through manual human interaction. But before we dig into the exercises and results, let’s first cover what these two platforms are.
Tools
ChatGPT
“ChatGPT (Chat Generative Pre-trained Transformer) is a chatbot developed by OpenAI and launched in November 2022. It is built on top of OpenAI's GPT-3 family of large language models and has been fine-tuned (an approach to transfer learning) using both supervised and reinforcement learning techniques” - Wikipedia (ChatGPT)
ChatGPT is essentially the most highly trained chatbot we have access to at this time. ChatGPT pulls it’s best guesses from over 175 billion pieces of pre-generated data-logic trees in memory (making up about 800gb across a cloud compute ecosystem). Training data originates from over half a trillion data-sets, and is pulled from these locations, commonly available to the public:
From OpenAI
410 billion sets of training data from Common Crawl, a non-profit providing datasets directly from the internet, with 60% of the learning occurring here, as the internet is a keystone for ‘natural language’.
19 billion sets of data from WebText2, which is essentially a Reddit trawler used to Scrape URLs from all Reddit submissions up to December 2017 with 3 or higher score. This data is key to how people communicate to each other on a popular forum, weighted at 22%
12 billion sets of data from Books1, this library is essentially the BookCorpus data set that has been used for machine learning, utilizing over 7000 unique novels that are openly available, accounting for 8% of the weight.
55 billion sets of data from Books2, previously unpublished but generally thought to be a new branch of LibGen, a peer to peer scientific journal sharing platform, accounting for 8% of the weight on scientific discourse.
3 billion sets of data from Wikipedia, primarily from the English wikipedia, but only accounting for 3% of ChatGPTs logical weighting.
So, TLDR; we have the most advanced natural language AI available at this current time, and ChatGPT 3.5 is around the corner.
MidJourney
So we have the text down, and content is king; but we are missing the visual models and views that make these imaginative stories more real and substantial.
“Midjourney is an independent research lab that produces an artificial intelligence program under the same name that creates images from textual descriptions, similar to OpenAI's DALL-E and Stable Diffusion. It is speculated that the underlying technology is based on Stable Diffusion.” - Wikipedia (Midjourney)
Screenshot (not mine) of Midjourney on Discord. Using “imagine [prompt]” to create and refine/upscale images.
Created by the Leap Motion co-founder, once an AR (augmented reality) skunk-works, pivoting toward making amazing visualization AI’s like DALL-E and Stable Diffusion more accessible via Discord, a communication and collaboration platform commonly used by gamers and developers. DALL-E and Stable Diffusion actually use a base-library from ChatGPT to help with natural language requests, so interoperability is already prevalent. Additionally the data-sets that are used by Midjourney are similar to ChatGPT, such as Common Crawl, with over 5 billion images with textual context. Ideally, the base language model compatibility should allow for good results in both text and visual form, but definitely doesn’t have the granular detail of a Visio model for contextual to vector information.
The Exercises
So here we are with powerful neural networks utilizing massive cycles of machine learning, pulling from more data a person could possibly parse in a lifetime. How can we apply these technologies to the rigor of Enterprise Architecture and it’s associated sub-domains? Can these tools provide clear roadmaps? Can they provide information on technical debt? Can they catch risks in the details, that a human may have missed? The applications are endless, but let’s focus on some exercises that mimic EA work at varying degrees of altitude, and see if we can utilize any results.
High - Digital Transformation from Ecosystem X to Ecosystem Y, what are the order of operations, strategic risks, and average cost from on-premise to cloud?
Medium - Let’s do a technology bake-off, in this regard, Infrastructure Automation platforms, libraries, and applications. What is the best option for a Fortune 100 Financial company? What are the lower level challenges such as Python library performance?
Low - UML Model, let’s challenge the visuals, can we take a UML model and convert it to a different view? E.g. Conceptual to Component diagram? Business Context to Infrastructure? Can ML help fill the gaps? Can AI create compelling and realistic architecture models?
Low - Can AI interpret syntax conversion, ontology conversions, framework differentiation, or tradeoff analysis (i.e. SEI ATAM)?
Approach
We’ll feed the prompts as written directly into ChatGPT and post the exact message back.
We’ll feed the exact prompt into Midjourney multiple times. Once with the exact prompt, then somewhat simplified, and ultimately try to get a simplistic statement that represents our expected outcome.
Synopsis score for accuracy, from experience, on a 10/10 scale.
Let The Fun Begin…
Digital Transformation (High)
Prompt: “Digital Transformation from a mainframe ecosystem to a cloud ecosystem, what are the order of operations, strategic risks, and average cost from on-premise digital transformation to cloud for a company with 30,000 employees in the insurance sector?”
ChatGPT Reply:
“The order of operations, strategic risks, and average cost of a digital transformation from a mainframe ecosystem to a cloud ecosystem for a company with 30,000 employees in the insurance sector can vary depending on a number of factors, including the specific goals and requirements of the company. However, here are some general considerations:
Order of Operations:
Define the business objectives and expected outcomes of the digital transformation.
Conduct a comprehensive assessment of the current IT infrastructure and identify the applications and data that need to be migrated to the cloud.
Develop a migration plan that outlines the sequence of applications and data to be moved to the cloud, as well as the timelines, dependencies, and risks involved.
Choose a cloud service provider and the appropriate cloud deployment model (public, private, or hybrid) based on the company's specific needs and preferences.
Conduct a proof-of-concept (POC) to validate the cloud deployment model and assess the performance, security, and scalability of the cloud environment.
Migrate the applications and data to the cloud in phases, starting with the least critical and complex ones.
Test and validate the applications and data in the cloud environment to ensure they meet the desired performance, security, and compliance requirements.
Train the employees on the new cloud environment and the tools and processes they need to use.
Strategic Risks:
Security Risks: Migrating to the cloud involves the transfer of sensitive data and applications to a new environment, which can increase the risk of security breaches if not managed properly.
Compliance Risks: Insurance companies are subject to various regulatory requirements and standards, which need to be met when migrating to the cloud.
Vendor Lock-in Risks: Choosing a specific cloud service provider and deployment model can limit the company's flexibility and agility in the future.
Cultural Resistance Risks: Resistance from employees to change can lead to a delay in the adoption of the new technology.
Average Cost:
The average cost of a digital transformation from on-premise to the cloud can vary widely depending on several factors, including the size of the company, the complexity of the applications and data to be migrated, the choice of cloud service provider and deployment model, and the extent of customization and integration required. However, here are some rough estimates:
Assessment and Planning: $500,000 to $1,000,000
Proof-of-Concept: $50,000 to $150,000
Migration: $10,000 to $20,000 per application
Testing and Validation: $50,000 to $100,000
Training: $100,000 to $200,000
Ongoing Maintenance and Support: $200,000 to $500,000 per year
These are rough estimates and may vary significantly based on the specific circumstances of the company.”
MidJourney Samples:
Synopsis
8/10
ChatGPT’s strategy order response is good but lacks some in-between nuance prevalent in cloud migration exercises, particularly the 6 R’s (Re-host, Re-platform, Re-factor/Re-architect, Re-purchase, Retire and Retain). The cost and prices are pretty damn accurate, surprisingly so, even the differentiation in cost and cloud bills are pretty close to what I see monthly in a Fortune 150.
UML diagram mainframe to cloud migration - Upscaled
Now the visuals are where things get wonky but also interesting. First, and most apparent, visual AI does a poor job at creating graphical typology (e.g. titles / headlines), so we can discount any gibberish that occurs. Now the visualizations themselves are interesting, in this set of 3 I started to pair-down language, starting with the full prompt and reducing language as I moved along, without variation or upscaling. The left most images definitely reprise the role of a general infographic regarding statistics for marketing, as we move right, we start to see graphics, but seems to be a time delineation on prevalence of cloud, of note is the great red/black/white (Ed Tufte standards!) visualization that clearly shows an iterative approach to hybrid cloud and expansion. As we move completely right we have the prompt simplified to “UML diagram mainframe migration to cloud” and we see shapes and patterns start to emerge.
If we upscale a particularly interesting generated image, we can see the underlying systems, a treemap indicating variance in libraries and technologies, and several integration interfaces going between a cloud layer and on-premise; indicative of the iterative journey from legacy on-prem to hyrid to cloud, this being a hybrid visualization that’s apt to the prompt.
Generally, the text response was most accurate with a surprising cost breakdown, but the pattern emergence from the visuals are the most intriguing.
Technology Bakeoff/Comparison (Medium)
Prompt:
“Let’s do a technology bake-off, in this regard, Infrastructure Automation platforms, libraries, and applications such as Ansible, Puppet, and Chef. What is the best option for a Fortune 100 Financial company to pivot traditional infrastructure management to infrastructure automation? What are the lower level challenges such as Python library performance, and what is the best platform to satisfy this criteria?”
ChatGPT Response:
“When it comes to infrastructure automation platforms for a Fortune 100 Financial company, Ansible, Puppet, and Chef are all viable options with their own strengths and weaknesses. In general, these tools are designed to help organizations automate the process of provisioning, configuring, and managing their infrastructure, which can improve efficiency, reduce costs, and increase consistency and security.
To determine which option is the best fit for a specific organization, several factors should be considered, such as the company's existing infrastructure, the skills and expertise of the team, the size and complexity of the organization's IT environment, and the level of customization and control required.
Here are some general considerations for each platform:
Ansible: This is a popular choice for organizations looking for a lightweight, easy-to-use platform that supports a wide range of IT environments, from small to large. Ansible uses a declarative language and YAML syntax, making it relatively easy to learn and use for both developers and operations teams. Ansible can also integrate with a variety of other tools and technologies, such as cloud platforms, networking devices, and databases.
Puppet: This platform is known for its scalability and flexibility, making it a good choice for larger organizations with complex IT environments. Puppet uses a declarative language and a domain-specific language (DSL) called Puppet DSL, which may require a steeper learning curve for some users. However, Puppet has a large community of users and a robust ecosystem of modules and plugins, making it easy to extend and customize.
Chef: Chef is a powerful and flexible platform that can handle complex automation tasks. It uses a declarative language and a domain-specific language (DSL) called Chef DSL, which may require more advanced technical skills to use effectively. Chef also has a large community of users and a wide range of modules and plugins available, making it easy to integrate with other technologies.
Regarding lower-level challenges, Python library performance may be a consideration, particularly if the organization is dealing with large and complex infrastructure. While all of the platforms mentioned support Python, they differ in their performance and efficiency. Therefore, if Python performance is a critical factor, it may be worth exploring each platform's performance benchmarks and documentation to determine which option is the best fit.
Overall, the best platform for a Fortune 100 Financial company to pivot traditional infrastructure management to infrastructure automation depends on the specific needs and requirements of the organization. Careful evaluation and testing of each platform is necessary to determine the best fit.”
MidJourney Samples:
Synopsis
4/10
ChatGPT’s response is orderly and precise in it’s details of differentiation particularly around the syntax knowledge and challenges. It does however state Python as fully compatible across the selections, but this isn’t the case as each has different compatible versions. It does however do a good job on calling out the python interoperability and notably calls out the performance challenges in this space, correctly pointing the analyst to deeper level documentation.
Now the images are where things get really wonky. We see a slew of images, the marketing stuff being hilariously a lot of men with arms folded, a common pose for a fed-up infrastructure person; the chef vs. puppet comparison with actual chefs and puppets; and pseudo-graphs showing some movement, and then the artistic representation with traditional infrastructure hooked up to clouds and wires. None of these images help us, though they do give us fun conceptual images to share.
UML Model Translations (Low)
Prompt: “UML Model, let’s challenge the visuals, can we take a UML model and convert it to a different view? E.g. Conceptual to Component diagram? Business Context to Infrastructure? Can ML help fill the gaps? Can AI create compelling and realistic architecture models? Show me an example of a business context diagram displayed as a conceptual diagram.”
ChatGPT Response:
Yes, it is possible to take a UML model and convert it to a different view. UML provides a range of diagrams for different purposes, including use case diagrams, activity diagrams, class diagrams, and component diagrams, among others. Each diagram represents a different view of the system being modeled and focuses on different aspects of the system.
Machine learning (ML) can help in the process of converting a UML model to a different view by automating some of the tasks involved in the transformation. For example, ML can help in the detection of patterns and relationships in the UML model, which can be used to generate new diagrams that represent the same information in a different way. However, ML cannot replace human expertise and understanding of the system being modeled.
AI can also help in the creation of architecture models by generating new diagrams based on existing models or by creating new models from scratch. However, like ML, AI is not a substitute for human expertise and understanding of the system being modeled. AI-generated diagrams may be compelling and realistic, but they still need to be validated and refined by human experts.
As for an example of a business context diagram displayed as a conceptual diagram, it's important to note that a business context diagram is already a type of conceptual diagram. It represents a high-level view of the system being modeled, focusing on its external actors, such as users, customers, suppliers, and other systems, and the interactions between them. A conceptual diagram, on the other hand, is a more general term that refers to any diagram that represents a high-level view of a system or a concept, without going into detail.
That being said, if you have a specific business context diagram that you would like to see displayed as a conceptual diagram, I could provide an example based on that.
Sure, here is an example of a business context diagram displayed as a conceptual diagram:
<imgur removed>
This diagram represents a hypothetical e-commerce system. The outermost circle represents the external actors that interact with the system, such as customers, suppliers, and delivery services. The middle circle represents the system itself, with its main components, such as the web interface, the database, and the payment gateway. The innermost circle represents the sub-components of the system, such as the login module, the product catalog, and the order management system.
This diagram provides a high-level view of the system and its components, without going into detail about their internal workings or interactions. It is a conceptual diagram because it represents a high-level concept or abstraction of the e-commerce system, without going into the technical details of its implementation. This type of diagram is useful for communicating the overall structure and purpose of the system to non-technical stakeholders, such as managers, customers, and investors.
MidJourney Samples:
Synopsis
6/10
ChatGPT’s response to a set of complex questions was pretty spot-on, particularly around the underlying fact that a Business Context and a Conceptual Model, are both conceptual. The ML response around view changes reflects common EA taxonomies, as they are meant to be interdependent across views. Great response, and a fun primer for ML + UML use-cases.
Where it continues to get interesting is the image synthesis from Midjourney. The overall pattern is accurate in that it reflects a key tenant of Business Context, a centralized solution; additionally the color coding pattern represents key distinctions in either infrastructure or people groups. Bonus points for human validation, though it may just be placating us /s.
Ontological Conversions and Tradeoffs (Low)
Prompt: “Can AI interpret syntax conversion, ontology conversions, framework differentiation, or tradeoff analysis (i.e. SEI ATAM)? What are the ATAM tradeoffs for implementing full active-active infrastructure vs. ethereal computing?”
ChatGPT Reply:
“AI can potentially interpret syntax conversion, ontology conversions, framework differentiation, or tradeoff analysis depending on the specific task and the quality of data available for training. However, it is important to note that AI is a tool and not a solution, and human expertise is still required to interpret and make sense of the results generated by AI algorithms.
Regarding the ATAM tradeoffs for implementing full active-active infrastructure vs. ethereal computing, it would require a more specific understanding of the context and requirements of the system in question to provide a comprehensive answer. In general, some potential tradeoffs that could be considered are:
Active-active infrastructure typically requires more hardware and network resources than ethereal computing, which could lead to higher costs.
Ethereal computing may be more susceptible to security risks since data is distributed and transmitted over a network, whereas active-active infrastructure can provide more control over the physical infrastructure.
Active-active infrastructure could provide better performance and availability since it allows for distributed processing and redundancy, while ethereal computing may have limitations due to network latency and bandwidth constraints.
These are just a few examples, and the specific tradeoffs would depend on the particular system being designed and the goals and constraints of the project. A systematic approach like the SEI ATAM (Architecture Tradeoff Analysis Method) can help to identify and evaluate the tradeoffs and make informed decisions about the architecture design.”
MidJourney Samples:
Synopsis
4/10
SEI ATAM is a methodology pioneered at Carnegie-Melon to assess and pivot designs to it’s most critical components, a relatively niche methodology employed most commonly by large industrial programs such as military equipment. ChatGPT’s response was pretty spot on, even with the usage of ethereal vs. ephemeral computing, as ephemeral is the most commonplace term e.g. network port assignments.
The assessment itself was accurate, and the grammar used comparison language in a startlingly manner. It did lack some details, perhaps if I referenced ‘ephemeral’ the results would vary, but generally this is too high level without good ontological explanation. Light on the ethereal and too heavy on the traditional.
Midjourney had interesting results, particularly the diagram of a building and associated callouts, something that would work well as an analogy for ATAM. The art piece with the servers are interesting in that it does truly show a blend of ‘ethereal’ and traditional data centers, a fun visual cue. Lastly, the graph charts are ultimately useless, but one can find some patterns, particularly around the scale usage on the bar graph and the uptick and adoption line graph, assuming showing lower cost for ephemeral/ethereal as it’s based purely on usage and has more accurate compute sizing due to it’s adaptive nature.
Conclusion
Pros
Natural language ChatGPT is incredible, it writes accurate English language grammar, stories and explanations.
Ignoring the gibberish typology from Midjourney, the patterns that come out are intriguing, and I believe this is the best place for EA to operate and expand upon, Architectural Patterns.
Metaphorical or analogical illustrations, anything you can imagine and refine; a great teaching tool for visual learners.
When doing technology comparisons it is incredibly powerful at force multiplying an individuals analysis efforts.
When challenged to provide metrics or dimensions, does a pretty great job.
Cons
ChatGPT can sometimes operate at too high a level, with lengthy introductions.
Both platforms pull from billions of records, whether those records are right or wrong in context. A great amalgamation, but be wary of ‘fruit of the poisonous tree’.
Midjourney can’t do typography at this time, and as such lacks a large degree of granular context needed for architectural models.
Possible Use Cases
There are a myriad of use cases for this technology suite, but for EA there are a few easy ones where we can find value.
The most prevalent use case with accurate answers are around technology bakeoffs or comparisons. Why limit yourself to Gartner, Forrester, or McKinsey when you can have an AI do all that heavy lifting to supplement your findings?
Cost estimations are great, it’s your private economist if used correctly; as the risk correlations are chatty and data is prevalent.
Architectural Patterns, since the underlying technology is pattern recognition, utilizing the pixel-perfect ML cycles can have some intriguing results.
Closing Statement
Incredibly powerful tools made easily accessible. The world is right to wonder about where this technology currently is, and where it’s going. The job of an Enterprise Architect is definitely safe at this time, the evolving nuances of an individuals experiences cannot currently be replaced by these awesome technologies.
Will I use these technologies in my everyday EA life?
Not sure, but it’s definitely intriguing, and worth your time. Maybe a 3rd eye would be handy.