Devlog
Translated from Spanish by 🌐💬 Aphra.
Published Aug 5, 2024
For quite some time, an idea has been bouncing around in my head: To achieve automatic translation into other languages for the few articles I publish on the blog.
More than one person might've raised an eyebrow: "Don't browsers already do that on demand?"
And they do. But I'm looking for something more complex. I don't want certain terms to be translated literally, and I'd like to add a sort of translator's notes that provide the appropriate context.
Until a few years ago, achieving something like this would've been extremely difficult. But nowadays, I believe it's feasible thanks to the emergence of generative language models.
I think the time has come. It'll be something small that I can complete in some free moments. And I want to share the technical decisions here in case anyone's interested.
By the way, it already has a name: 🌐💬 Aphra.
Published Aug 6, 2024
When starting a new project, it's ideal to look for anyone who's previously done something similar. This'll help you situate your development, gather ideas, and decide on what points you're going to differentiate yourself.
I'm clear that I want to combine different calls to language models to achieve a flow that's currently being called "agentic". The first article I read on the subject is Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models, abbreviated as STORM. It proposed a combination of calls to language models to achieve, starting from the user entering only a topic, a Wikipedia-type article with real references. It opened my eyes to how the limits of these models could be pushed further, simply by letting them perform tasks with an approach more similar to what a human usually follows. By the way, if you want to see an implementation of this paper, you have one made with LangGraph at this link.
For me, this represents a new programming paradigm. It doesn't replace any at all, but rather covers a new need in which we can afford to automate certain tasks previously unthinkable for a machine to perform. To the point where we have companies basing their business on a complex flow, keeping that use of models and prompts hidden as if they were the Coca-Cola formula. In fact, recently a Spanish startup formed by two people sold that secret (and the entire company of course) for no less than about 12 million (approximately, the data isn't yet public as far as I know) having only 5 months of life. In this case, the project, flow, and prompts of 🌐💬 Aphra will be open source.
The next reference is much closer to what I want to do, and it's this recent proposal from Andrew Ng's magnificent team. This project, published on Github, demonstrates how a translation improves simply by making an extra call to a model to reflect on it. I consider that flow my starting point, but I think for my use case it falls a bit short, as it doesn't provide an updated context on the terms that appear, nor does it generate notes that help understand the translation decisions. Therefore, the use of 🌐💬 Aphra will be more expensive, as the information goes through more stages. But I hope the difference is worth it.
Source: https://www.linkedin.com/pulse/aphra-devlog-1-referencias-david-romero-santos-54zrf
Published Aug 8, 2024
Probably, designing the flow is the most complex task of this project. Then there are other very important decisions, such as the specific prompts and models to be used, but these are options that are easier to modify later. The chosen flow will greatly influence the usefulness, cost, and latency of the translation.
The problem is that we still know very little about this technology to face the design with any guarantee. But that's also the beauty of it, that for now, we're just here to play. Relying on intuition. The general advice is to think about how a human would approach the same task and, from there, divide the work into well-defined steps, avoiding redundancies. I've never faced a translation job, so we're starting quite badly in this aspect. However, if I myself had to translate an article from my blog, the steps I'd take would be more or less these:
- A first reading to understand the article.
- Search for information about those terms or expressions I don't understand and entities I don't know.
- Perform a translation using my words, trying to maintain the original style.
- Look for someone native in the language I'm translating to, to make observations about things that don't sound good or aren't adequately understood.
- Based on the criticism received, correct the translation.
Well, more or less... I'd have to implement something similar to that with language models. And so we have the following diagram with the initial operation of 🌐💬 Aphra, which I detail below.
Step 1 - Analysis
A language model (which we'll call Writer LLM) analyzes the article, indicating which expressions, terms, or entities may have problems when making a literal translation to another language.
Step 2 - Search
Another language model with Internet access (which we'll call Searcher LLM) receives the analysis resulting from step 1 and searches for information about each of these terms, entities, or expressions, thus forming an updated context.
Step 3 - Initial Translation
Parallel to steps 1 and 2, a Writer LLM performs a translation of the original text, trying to maintain the author's style (without access to the analysis and information obtained in step 2).
Step 4 - Critique
A third language model (called Critic LLM) receives the original text, the basic translation from step 3, and the information obtained in step 2. Having access to all the updated context, it'll make an informed critique of the decisions made in the translation from step 3.
Step 5 - Final Translation
Finally, a Writer LLM receives all available elements (original text, basic translation, updated context, and critique) to write the final translation, taking into account the improvement comments received in step 4.
Well... What do you think? 😅
Time to put it to the test!
Source: https://www.linkedin.com/pulse/aphra-devlog-2-dise%25C3%25B1o-del-flujo-david-romero-santos-ttfpf
Published Aug 10, 2024
Finally, the time has come to subject 🌐💬 Aphra to a comparative evaluation with other available options.
I don't intend to do an exhaustive analysis, but rather perform some preliminary tests that'll help me decide if it's worth publishing the project. Keep in mind the following in these tests that I'm going to perform:
- Each model execution involves some randomness. Therefore, it's possible that two executions with the same input produce different outputs.
- When presenting results prior to tool launches, cherry-picking is usually done, that is, the best results are chosen from several attempts. In my case, I'll always stick with the first output I get.
I'm going to perform the first test on my article "Teachers, lay down your arms against AI".
The simple translation, that is, the one performed only with a prompt and a call to the model, you can see at this link. The total cost of this translation has been €0.05.
The translation performed by 🌐💬 Aphra is available at this other link. In this case, the total cost amounts to €0.25.
Although the references added at the end are evident, I understand that it can be difficult to notice to what extent the translation improves in the main text. That's why I've prepared this comparison for you (the first translation is the original, and the second is the one performed by 🌐💬 Aphra).
Apart from the additional references, which I consider an excellent contribution, there are no drastic changes in the text. However, I find the adjustments made very appropriate. Somehow, the translation uses expressions more akin to what I would probably have chosen if I had done it personally.
Now, let's compare with translation-agent by Andrew Ng (the competition 😜). In this case, after modifying the code to use the same model I used (making the comparison more equitable), this result is obtained. The total cost is €0.38. It's more expensive when it shouldn't be, and the explanation is simple: I've decided not to divide the translation into several blocks, as my articles are short enough not to exceed the maximum output tokens of the model I use. Thus, I simplify the technical decisions in this first version. In contrast, translation-agent does divide the text into fragments, which implies more calls and, therefore, a higher cost depending on the length of the text.
Let's move on to what interests us: here's the comparison (translation-agent as original and 🌐💬 Aphra as final). The references again only appear in the 🌐💬 Aphra version, which I consider an added value. Also, in a not too complicated text (something that is probably influenced by the prompting), I more easily recognize the style choices and expressions made by 🌐💬 Aphra, which are much closer to what I would have chosen. Maybe it's just suggestion... But you'll tell me your opinion.
Although we could stop here, I wanted to add one more example. Now it's time for a translation from English to Spanish. We're going to start from this text (very interesting, by the way), which is one of the letters that Andrew Ng sends in The Batch.
This is the simple translation (cost: €0.04) and this is the one performed by 🌐💬 Aphra (cost: €0.26). You can see the comparison here. I think the explanatory notes are very useful. Although the changes in the text are minimal, if you notice, it seeks to vary the repetition of words like "lluvia" (rain) and "intuición" (intuition), replacing them with other equally valid ones. This is very interesting, as it improves the quality of the text in Spanish.
Finally, here's the translation performed by translation-agent (cost €0.18). And the last comparison between both translations you have at this link (translation-agent as original and 🌐💬 Aphra as final). This time, I'll let each person assess the differences according to their criteria 😇.
Public access to 🌐💬 Aphra is getting closer every day... Do you have any suggestions?
Source: https://www.linkedin.com/pulse/aphra-devlog-3-hora-de-la-demo-david-romero-santos-qdpcf
Published Aug 14, 2024
How should 🌐💬 Aphra work? Using models that can be run locally or those that run in the cloud?
Let's do a quick comparison.
If we opt for 🌐💬 Aphra to work locally, there are some advantages. By running on the user's equipment, you don't depend on errors or model changes made by the provider. If the information you need to translate is personal, privacy is absolute. Also, the cost of these models is limited to the electricity consumption of your equipment when running them. However, there are also drawbacks. For example, setup becomes more complicated, as it'll be necessary to download and configure those models beforehand. The user's equipment and storage will have to be powerful enough, which increases the entry barrier. Also, you won't be able to test 🌐💬 Aphra at its full capacity, as the quality of translations will be affected by not being able to use the most powerful models currently available.
On the other hand, if we decide that 🌐💬 Aphra uses models executed in the cloud (someone else's computer), we find certain positive points. We'll be able to use SOTA models, obtaining the best quality in translations. Testing one model or another will be as easy as changing a line in the configuration, which allows for faster iterations and tests. Also, it can run on any equipment, regardless of its capabilities, as only an Internet connection will be needed. However, there are also disadvantages. Along with the loss of privacy of the information we want to translate, we risk encountering errors when calling an external API, which might not be available at certain times. Also, we'll need an API KEY, which will have an associated cost per use.
The choice is simple if we don't lose sight of the initial objective of the project: to automate translations of a personal blog. The most practical way would be to incorporate 🌐💬 Aphra into the publication pipeline. If we use cloud models, this is trivial. However, with local models it's not possible, unless we have a very powerful computer always on, which would significantly increase the costs of keeping the system active.
Therefore, for the first version of 🌐💬 Aphra, I'm opting to use cloud models, without ruling out that in future versions local models could be used as an alternative.
What specific models will be used? It'll be discussed in the next post 😉.
Source: https://www.linkedin.com/pulse/aphra-devlog-4-local-o-nube-david-romero-santos-yhzse
Published Aug 15, 2024
The list of available language models grows day by day. It's practically impossible to keep up with new publications. Some models present small improvements in specific areas, others are larger, some have good capabilities but are small, and from time to time we see quite significant advances in certain aspects. Staying up to date has become a real madness.
To top it off, nowadays we have a considerable problem: it's becoming increasingly difficult to make an objective comparison between models. When a new model comes out, how do we know how good it is compared to the rest? We have traditional benchmarks like MMLU, but they're becoming increasingly obsolete. In fact, when GPT-4 was launched, it was compared with its ability to solve certain exams designed for humans. For those interested in this topic, it seems that the SWE-bench benchmark will be the most common reference point for next-generation models. This benchmark is designed to check how many real GitHub problems a language model can satisfactorily solve, which is a good measure, as the next models will be focused on the ability to reason and execute actions in the real world. This is confirmed by OpenAI in one of its latest publications.
Let's return to our problem. A website I often use as a reference when I have to choose a model is lmarena.ai. Its authors had a very accurate idea: allow users to write an input, generate two outputs with two different models, and have the user vote which is better (or if they are equal). Only after voting are those models revealed. It works so well that many organizations are testing their models there to have a reference before making them public. Recently, they have also created different categories of prompts to make the comparison more specific. This system is not infallible, of course (humans have different criteria for defining what is better or worse), but at least it provides a starting point to get an idea.
In recent months, the top positions in this "arena" have been occupied by proprietary models from OpenAI, Anthropic, and Google, with Meta and its open-source models always pushing to reach the podium. Some weeks, one model moves ahead, although OpenAI usually manages to stay longer in first place.
If you remember the design of 🌐💬 Aphra, we had three language models: the Writer, the Searcher, and the Critic. The choice of specific models is not crucial. And it's not because 🌐💬 Aphra is programmed to work with OpenRouter, a cloud model provider. This platform is one of my favorites because it acts as an intermediary proxy between the vast majority of available models, without increasing their official cost. You load your account with credits and, with a single API KEY, you can change the model by simply modifying one line of code. In the case of 🌐💬 Aphra, changing one line in the configuration file. The connection is compatible with the OpenAI API, which is already beginning to become a standard. It's all advantages. So each person who uses 🌐💬 Aphra can choose the models they prefer for each case.
What I am going to tell you are the models I used in the demo I showed. The Searcher model, since it needs an Internet connection, was pretty clear. In OpenRouter, the only ones with a connection are those from the Sonar family by Perplexity, so I chose the 70B one. The performance difference between the 70B and the 405B is not so noticeable, but the price of the latter is five times higher. Why does the existence of the 405B make sense then? I leave the question open in case someone wants to investigate and answer in the comments 😇. I could have also programmed an agent behavior with access to a tool for Internet connection, and thus use any model. But that would also entail an additional API KEY and a bit more complexity in the code. Perhaps this option will be available in a future version. For the Writer and Critic models, really any of those at the top of the ranking would work for us. In general, I don't like how Google's Gemini models write. Although, in fact, the only project I have in production (secret for the moment) uses Gemini, but because the context size is absolutely exaggerated, which suits me in that specific use case. OpenAI models are fine, but the writing style is quite formal. In fact, at this link you can see how translate-agent performs the translation of the last article on my blog, by default using GPT-4 (€1.07 cost, by the way). From the few tests I've done, the current model I like best for creative writing is Claude Sonnet 3.5 from Anthropic, so that's what I chose for both Writer and Critic. I think it's the one that best maintains the casual style I try to maintain in the articles.
Well, one less secret revealed. There's less and less time until the publication of 🌐💬 Aphra 😁.
Source: https://www.linkedin.com/pulse/aphra-devlog-5-escogiendo-llms-david-romero-santos-w3nlf
Published Aug 16, 2024
I've tried to design 🌐💬 Aphra so that testing it, incorporating it into your own project, or modifying it is as simple as possible.
The first contact is as easy as accessing a Space on Hugging Face. There you'll find a Gradio interface. You only need to enter an API Key from OpenRouter, choose the models you're going to use, the source language and the target language. From there, you just need to upload a file or copy and paste a text to test the translation with 🌐💬 Aphra with a simple click.
If you prefer to try 🌐💬 Aphra on your own computer, you have all the usual options available: global installation with pip, with Poetry, in a Python virtual environment, or using Docker. In all cases, you'll need to clone the project (or download the release) and create the config.toml file with the OpenRouter API Key and model selection. Everything is explained step by step in the repository's README.
The aphra package contains a folder where the .txt files with the different prompts for each of the flow steps are stored. It also includes the following Python files:
- The classic init.py
- llm_client.py (manages calls to language models in OpenRouter)
- prompts.py (takes care of formatting the .txt files of the different steps into text strings, substituting the corresponding variables, such as source and target languages)
- parsers.py (auxiliary functions to extract relevant content from language model responses)
- translate.py (the main file containing the translate function, which you can incorporate into your own code)
Using 🌐💬 Aphra in any project is as simple as the following:
from aphra import translate
translation = translate(source_language='source_language',
target_language='target_language',
text='text_to_translate',
config_file='path/to/config.toml',
log_calls=False)
print(translation)
As you can guess, if log_calls is set to True, a file will be generated with the prompt and output of each of the models used.
Of course, in the repository there will be an explanatory README file (which will also be deployed through a GitHub Page), as well as a Wiki. Within it, there's a link to documentation of each and every function and its parameters, automatically generated with pdoc.
And not much else. If all goes well, 🌐💬 Aphra will be published tomorrow, as I've heard that Friday is a bad day to go into production 😉.
Source: https://www.linkedin.com/pulse/aphra-devlog-6-c%25C3%25B3mo-se-estructura-david-romero-santos-uaarf
Published Aug 17, 2024
🌐💬 Aphra is now published.
Before sharing the corresponding links, I'd like to explain where the name Aphra comes from. Aphra Behn was a writer who developed her work during the 17th century, becoming the first professional female writer in English history. But not only that, she also dedicated herself to making translations to earn a living, allowing works in French or Latin to be available in English. With this name, I intend to pay tribute to her figure, about whom Virginia Woolf wrote:
All women together, ought to let flowers fall upon the grave of Aphra Behn... for it was she who earned them the right to speak their minds
You can access all the information and code of 🌐💬 Aphra in the official repository.
I hope you like it and, of course, that you're encouraged to contribute
Source: https://www.linkedin.com/pulse/aphra-devlog-7-en-producci%25C3%25B3n-david-romero-santos-nwp9f
Published Jul 7, 2025
It's been roughly a year since I published 🌐💬 Aphra. It was born to address a need: automatically handling English translations of articles on my blog davidlms.com.
I'm very satisfied with the result, but I believe it has much greater potential. Its architecture, until now, fulfilled its purpose. However, it has turned out to be a significant bottleneck in its capacity to grow.
Today I want to announce that the new 🌐💬 Aphra 2.0 is ready. Its main novelty? It has evolved into a modular system. This means that, from now on, new translation pipelines can be added independently for other types of content beyond short articles. I'm currently developing one that will allow for practically professional translation of subtitles, regardless of their length. Thinking through and prototyping the design of said pipeline is proving to be quite fun.
The new structure is very simple:
In aphra/workflows you'll find the available workflows. Currently, only the original one for short articles. In aphra/prompts/* you can access the prompts corresponding to each workflow.
This way, you have different options at your disposal: from simply customizing the prompts, modifying an existing workflow, or creating a new one from scratch.
Feel like designing one? 🌐💬 Aphra 2.0 awaits you in its repository.
Source: https://www.linkedin.com/pulse/aphra-devlog-8-lista-para-crecer-david-romero-santos-htvff