R. Larraz, CEPSA, Madrid, Spain
In the artificial intelligence (AI) segment, large language models (LLMs) like ChatGPT, Gemini, Co-pilot, Perplexity, Claude, et al., have emerged as powerful tools for processing and generating human-quality text. These sophisticated algorithms, trained on massive datasets of text and code, possess the ability to comprehend and produce language in a wide range of contexts. However, despite their capabilities, LLMs also present a unique challenge: how to effectively access and harness the vast knowledge they hold.
Jorge Luis Borges' short story, "The Library of Babel," describes an infinite library containing every possible book ever written (or to be written). While this repository of knowledge might seem like a utopia for information seekers, it also presents a paradox—the sheer volume of information makes it virtually impossible to find anything specific. LLMs, in many ways, mirror the paradox of Borges' library. They contain an immense store of information, but extracting and utilizing this knowledge can be a complex task. The challenge lies in effectively communicating with language models, guiding them towards the desired information and prompting them to generate meaningful outputs.
This is where prompt engineering comes into play, as shown in FIG. 1. Prompt engineering involves carefully crafting instructions that guide LLMs towards specific tasks or desired outcomes. By employing prompts, users can transform LLMs from repositories of raw data into versatile tools for generating creative text formats, translating languages, writing different kinds of creative content, and answering questions in an informative way.
Prompt engineering is an emerging discipline at the intersection of AI and process engineering and will undoubtably contribute to the hydrocarbon and green molecules processing industries. Parts 1 and 2 (October 2024) of this article explore and provide a glimpse of how this technique can help process engineers and operators to manage the power of advanced AI models to optimize processes, contribute to solve complex problems, and improve operational efficiency in refineries and petrochemical plants.
The information included in this article is theoretical and does not correspond to any real process or situation. It is only shown to illustrate the potential capabilities of LLMs and should not be used in any way.
LLMS
LLMs are advanced AI systems that can understand and generate human-like text across a wide range of topics and tasks. These models are trained on massive datasets of text and code, enabling them to process and generate human-quality language with remarkable fluency and accuracy.
The architecture of LLMs. To illustrate the process and explain how LLMs work, consider the flowchart in FIG. 2.
LLMs operate on the principle of predicting the next word in a sequence based on the context of previous words.
A simplified breakdown of LLM technical functioning is listed here:
This process allows LLMs to generate coherent and contextually relevant text based on the input they receive. The immense scale of these models—often containing hundreds of thousands of parameters—enables them to capture complex patterns and relationships in language. The performance of the LLM can be adjusted by tuning some parameters, as shown in TABLE 1.
In essence, LLMs work by recognizing patterns in language, using these patterns to understand input, and then generating appropriate responses based on statistical probabilities learned from their training data.
Prompt engineering: Guiding LLMs to reveal their knowledge. Prompt engineering has emerged as an essential discipline in LLMs, serving as a bridge between the knowledge held within LLMs and the specific needs of users, enabling effective communication and unlocking the full potential of these language models. The prompts should meet the following characteristics to be effective:
Key techniques in prompt engineering. Prompt engineering techniques like, one-shot, few-shot, chain of thoughts and tree of thoughts (detailed below) are different approaches to optimize interaction with LLMs and achieve more accurate and relevant results. These techniques are based on the idea of providing the LLM with additional and structured information to guide its reasoning and improve its performance on specific tasks.
Many other techniques exist for prompt engineering, such as role prompting (the LLM is asked to take a specific role relevant to the task to solve), or autotuned prompting (the model is asked to construct the optimal prompts based on several requirements and objectives). The methods described above are the most common strategies to start and should be enriched to obtain successful results.
Prompt engineering in the context of processing hydrocarbons and green molecules involves the careful formulation of instructions or questions (prompts) for AI models. These prompts must incorporate industry-specific knowledge, relevant operational data and clear objectives to generate useful and applicable insights. Eventually, the key components of an effective prompt for the processing industry should be:
Additionally, the following aspects should be considered:
The risks associated with these processes should never be forgotten, and the prompts must include specific considerations about:
PROMPT ENGINEERING EXAMPLES
The following section shows some preliminary examples of how to apply one-shot, few-shots, chain of thoughts and tree of thoughts prompt engineering techniques to solve common problems in a petroleum refinery.
One-shot prompting. This approach provides a single instruction or example to guide the AI's response.
Example: "Provide three methods to reduce coke formation in a catalytic reforming catalyst."
Few-shots prompting. Here, multiple examples are offered to help the AI better understand the desired response pattern.
Example: "Given the following problem-solution format for process optimization, provide a similar solution for the third case":
Chain of thought prompting. This method asks the AI to show its reasoning step by step.
Example: "A crude preheat train shows a 15% efficiency drop. Analyze possible causes and solutions, showing your reasoning at each step”:
LLM answers:
Tree of thoughts prompting. This method breaks down complex problems into sub-problems and explores multiple solution paths.
Example: Select the best technology for producing sustainable aviation fuel (SAF) using the tree of thoughts strategy. Consider economic criteria in your evaluation:
LLM answers
Takeaways. While the answers provided in this article are debatable, they do provide a clear image of the capabilities inherent to LLMs and the role of prompt engineering as a relevant skill for hydrocarbon and green molecules professionals. The ability to formulate effective prompts will become an important tool and should be guided by experts' technical knowledge. The synergy between human expertise and AI can help the industry to address exciting challenges like the refinery of the future configuration.
Part 2 of this article (October 2024) will further discuss prompt engineering and extracting operational excellence knowledge from AI. HP
LITERATURE CITED
Rafael Larraz is a Senior Consultant at Cepsa and Coordinator of the Petroleum Section at the Spanish Institute of Energy. With more than 30 yr of experience in refinery operations and engineering, Dr. Larraz has been involved in numerous large-scale projects, especially for refinery expansion, as well as biofuels production projects. Dr. Larraz holds a PhD in chemical engineering from La Laguna University and an Executive MBA from the IESE Business School.