AI models that export free of charge are here, but the most critical number has not been made public

Read a real request first, and after you read it, you will know what this thing does.

You threw it the customer's exact words:

Help, my transfer has failed for three consecutive days.

Here are three questions, and the options are up to you:

1. is this news urgent? Which group should 2. be assigned to? Accounting/Technology/Sales How angry are 3. customers? Calm/dissatisfied/angry

What it returns to you is this:

The first question is whether it is urgent: 0.95.

The second question is to whom: Accounting, grasp 0.80. The respective probabilities of the three options are accounting 0.87, technology 0.13, and sales 0.

The third question, how angry youare: 1.04 points, grasp 0.94. The probability of being calm is 0, dissatisfied 0.96, and very angry 0.04.

There is no word, only three sets of numbers.

When your program gets "Accounting", it will directly put the work order into the queue of the accounting team, and when it gets 0.95, it will mark it urgently. There is no need to parse any paragraph of text in the middle.

This model is called Jev and was released on September 16. It comes from TypeSafe AI that has been invisible for two years. Founder Diogo Almeida is one of the co-inventors of ChatGPT, and the company received a $40 million seed round.

The most counterintuitive thing about it is that it doesn't write a word.

!

1. It can only do three types of questions

Understanding the above example means understanding all its skills. It only recognizes three types of questions:

Multiple choice questions. Pick one of the few options you give, up to 255. Returns the one you picked, plus the individual probability for each option.

True and false questions. Returns a number from 0 to 1, 0.95 is "95% probability".

Grading questions. If you give a few gears, it will give a point, which can be added to a decimal number. 1.04 It means "somewhere between dissatisfaction and very angry, obviously dissatisfied."

Apart from these three, it cannot do anything. Can't write summaries, won't respond to customers, and won't write code.

There are two other examples in the official document that show what the job it wants to take.

Refund review. Throw in the work order, order record, and refund policy together, and ask two true and false questions: Is the customer requesting a refund, and whether this should be refunded according to this policy. The order clearly stated that two US$49 payments had been deducted, and the policy stated that repeated deductions could be refunded, so it gave 0.99 and 0.98.

Account risk control. It took 12 days to open an account, failed to log in five times, and requested to change the password in a new country, but I also successfully logged in with commonly used devices. It gives a risk of 1.84 points (0 low/ 1 medium/ 2 high), and the third probability is low 0, medium 0.16, and high 0.84, and gives a score of 0.81 on "Do you want to switch to manual labor".

To put my own opinion: I think risk control is too conservative. Successful login of a commonly used device is a strong "not stolen" signal, but it is still down to 0.84. Of course, being suspicious in risk control is not a problem at all.

On the other hand, I think the customer service is quite particular: if the transfer fails, it will give 0.87 for the accounts and 0.13 for the technology. The failure to transfer money may indeed be due to the failure of the payment channel, so it didn't say anything to death.

2. Why is it fast and why the output is free

To explain this clearly, you must first look at how ordinary large models write.

ChatGPT writes a sentence, and the actual process is:

1. reads your entire question and forms a state inside the model
2. scores all the words in the word listbased on this status, and picks the one with the highest score and spits it out
3. connects the words that have just been spat outto the end of the question, calculates it from the beginning, and predicts the next word
4. Repeat step 3 until you finish writing

Step 3 is the root cause of all slowness and costliness. Writing 100 words requires repeating them 100 times. If you use a model that can "think", it will first write a large piece of reasoning in your mind, which is repeated hundreds or thousands of times, and every word will be charged.

Jev's process is:

1. reads the materials and options together to form a status. This step is exactly the same as the big model.
2. does not rate 100,000 words,only rates the three options you have listed: accounting 0.87, technology 0.13, sales 0
3. output. End.

There is no cycle. The answer is one selected from several options. It is not a word in itself and does not need to be taken back and recalculated.

!

For example.

A big model is like writing a composition. Write a word, look back at what was written in front of you, and write another one. Write longer and slower.

Jev is like writing an answer sheet. Read the questions and options together, and paint one directly.

The key is: the speed at which you apply the answer sheet only depends on whether you have finished reading the questions, and has little to do with the options.

Therefore, the nature of "export free" is not quite the same as everyone thinks. This isa structural necessity and has nothing to do with good intentions in pricing: it does not have the action of "writing", and naturally there is no link to charge per word. The official price list says "too cheap to meter", which is too cheap to measure.

To put it all in the way: Input still costs money,$0.042 per million tokens, and it is now the early access queue for applications, and ordinary people don't need it today.

3. Is it a new architecture

Let's be clear here: TypeSafe does not disclose its architecture. The official statement is not to talk about it for the time being, and there may be papers in the future. The developer community has speculated from text diffusion that the encoder adds classification headers, but there is no conclusion.

So the following is based on evidence and not the official statement.

Someone searched its billing data and found two places that did not match.

First, for a 255-option request, the bill says "2714 tokens were output", but it returns as fast as a request with only two options. It is impossible to write 2714 tokens so quickly.

The second one is more interesting. This "output token number" will change with the length of the problem identifier, and the official document states in black and white thatthis identifier will not be issued to the model and will not participate in reasoning.

How can something that the model has never seen before affect how much it writes? There is only one explanation: the number was calculated from the returned results according to the formula after the inference was over. It didn't even write.

This eliminates two possibilities: it is not a word-for-word generation, and it is not a diffusion model (diffusion requires repeated iteration, and 255 options cannot be as fast as 2 options). The rest is to read it in one go and read it directly from a classification header.

So is it a transformer? Almost certainly, and the skeleton is likely to be the same as the larger model.

Two reasons. First, billing presents the characteristics of a "shared prefix": ask a few questions for the same batch of materials, and only charge the materials once. This is a typical practice of caching what you have read and not rereading it, and this is a standard feature of transformers. Second, it must have quite solid world common sense to judge whether "failed transfer" should be attributed to accounting or technology, and this kind of knowledge can only be obtained through pre-training of massive text.

So the accurate statement is: the skeleton has not been changed, but the last layer has been replaced, plus the training method.

Here is another point that is easy to be confused, and I think it is worth saying specifically:

The last step in the big model is to make multiple choice questions. It predicts the next word, essentially making a "choose one out of 100,000 words" among the 100,000 words in the vocabulary.

What Jev did was to change the question from "Choose one hundred thousand" to "Choose one of the three you gave", and thendo it only once, without repeating it.

The person doing the reverse operation said something very pertinent: The output layer of the language model is originally a matrix plus a normalization. If you change it to a special classification header, you can't see the difference from the outside.

This is a beautiful engineering choice, but the official calls it "a new class of cutting-edge models." I think it's a big story.

4. How much do I save by "free output"? I did some calculations

I calculated this myself, assuming it's written here: Enter 2000 tokens for a decision-making task.

!

What did you use to make decisions?	Exports as a percentage of total costs	Change to Jev Province
Non-inferential models directly spit out JSON	1.2%	4.8 times
Non-inferential model (expensive file)	1.2%	24 times
Inference model (with thought chain)	54.5%	52 times
cutting-edge reasoning model	75.0%	952 times

The conclusion is a bit counterintuitive:

The selling point of "free output" is almost a hollow favor for non-reasoning scenarios. Because output only accounts for 1.2% of the cost, it is almost as good as not being exempted. What really saves money is the input unit price, which drops from the regular 0.2 to 10 dollars per million to 0.042 dollars.

Only when you are using an inference model to make hard classification, that is, asking a thinking model to write a large section of inference first and then give the answer, when the output accounts for more than half, can "free output" really be valuable.

I checked their accounts. The official demonstration of letting Jev play Doom cost 10 calls per second, about US$7 an hour, and discounts down to US$0.0001944 each time. The price is reversed at a unit price of 0.042, and about 4630 tokens are entered each time. The size of the game state is right, the pricing is self-consistent, and there is no false standard.

5. There is only one hard job they really do

Putting aside packaging, there are three practical things: parallel sampling (asking five questions at a time and calculating them at the same time), two-stage processing of more than 255 (scoring separately first and then picking), and a third thing.

The third thing is the only hard job: to train the number of "grasping" accurately.

What's so difficult about this? There is a picture in the GPT-4 technical report that says it very bluntly: for the pre-trained model, it is 80% accurate if it is said to be 80% sure; but after post-training, this ability has obviously deteriorated. The report's exact words were that post-training severely impaired calibration.

Translation into human language: While we adjusted the model to be able to chat and be obedient, we also ruined its ability to "know how much you are". It becomes pretentious.

TypeSafe's training method is specifically designed to break this vacuum: don't train it to speak beautifully, just train it to be accurate.

I think this direction is the right one, and it is a real problem. But here's the trouble.

6. The most important number was not given

Let's first explain why this number is tight.

The real usage of Jev is like this: if it says it is sure, the machine will do it directly; if it says it is not sure, it will pay for a large model to take a look.

Saving money depends on this gate.

Then the question arises: Is it accurate when it says "grasp 0.8"?

!

Make an account. Suppose you have 10000 work orders a day, set a rule: grasp automatic dispatch orders above 0.8 without anyone looking at them, and this type accounts for 90% of them, or 9000 orders.

If it says 0.8, it will be 80% correct, and 1800 of the 9000 are wrong. You know what you want, and you can make arrangements accordingly.

If it says 0.8, only 55% of the tickets are correctand 4050 are wrong. There are 2250 extra pictures, and you don't even know they exist.

In the customer service scenario, the price of dispatching the wrong team is that the customer waits two more hours and accepts it. But in the previous example of risk control,"what should have been transferred to manual work has not been transferred" means that a stolen account has been released. At this time, 20% and 45% are two different things.

So this number is accurate and directly determines where you dare to set the gate.

In the official blog, there areno calibration errors, no reliability curves, no verifiable numbers, only adjectives: "calibrated probability" and "the higher the confidence, the more accurate the".

7. What's more troublesome is that their test method cannot calculate this

This is what I care about the most.

Their evaluation method is officially written as follows: instead of optimizing the standard answers, it assumes that there is a correct calculation process, anduses the predictions of the largest, smartest, and most expensive external models as reference probabilities. Specifically, we take the average of the two models, Astra and Fable.

The problem is that those two models are also guessing.

Take other people's guesses as the standard answer, and what you measure is "Do you think they are like?"not "Are you right?"

As for the matter of "being sure or not", by definition, there must be a definite right or wrong: if you say that you are 80% sure, you have to know how many times you have actually been right to know whether this 80% is a lie.

Without the true answer, even if the correct rate cannot be obtained; if the correct rate cannot be calculated, the question of "whether it is accurate or not" cannot be answered fundamentally.

This is a question of the method itself, not an issue of attitude. They were actually quite candid and took the initiative to write a lot of reservations: the four test processes were written by people done by their own team, and there may be deviations, those multiples are the high-end of actual income, the reviews are run on notebooks on the West Coast, and there is no way to self-certify pricing without being subsidized. Some people in the developer community pointed out that in a picture posted by the official itself, Jev's original accuracy rate is lower than that of Sonnet 5.

By the way,"I won't hallucinate." This statement is only half true: if you only give it five options, it certainly cannot come up with a sixth one, but it can still choose the wrong one among these five. Being unable to make it up and being unable to make it wrong are two different things.

8. There is still a real pit

There is a saying in the official document that is easily skipped: each other cannot see each other's answers to several questions in the same request.

In the refund example, the two judgments of "whether you are requesting a refund" and "whether the policy supports it" are independent of each other, and there is no problem when asked together.

But if you continue to ask "How much money should I refund?" and this judgment depends on the previous conclusion, you have tore-issue the request and pass on the order, policy, work order and other materials again. The input is charged.

The meaning is clear: it is fast, provided that your judgments are independent of each other. Once your process is to "judge the type first, and then decide what to ask next based on the type", you have to do it round after round, and you will get a discount 200 times faster.

This is not a flaw, it is the inevitable price of parallelism. But when selecting a model, you must first see what your process looks like.

9. How to test it yourself after getting it

When you really get access, don't rush to take over production. This set of actions took less than half a day:

The first step is to save 300 to 500 historical data items from your own business that already have the correct answers. You already have records of whether the work order was missed, whether the refund was finally refunded or not, and whether the account was really stolen in the end.

The second step is to run the full amount and save the confidence value it gives along with the true right and wrong.

The third step is to divide boxes according to the grasp value. 0.5 to 0.6 a box, 0.6 to 0.7 a box, and so on. Calculate the actual accuracy rate in each box.

The fourth step is to draw a picture of "the certainty of what it says" and "the actual correctness rate". If the two basically fall on the same slash, it means that its confidence is credible, and you can safely set the gate according to the threshold. If it says that the box with 0.8 actually only has 0.55, then this model is trying to be brave in your scene, and the gate has to be lifted up, or simply don't use this automatic release method.

Step 5, use this graph to reverse your threshold. As much error rate as you can tolerate, go to the graph to find the corresponding confidence value instead of tapping your head to set 0.8.

The cost of this matter is very low, hundreds of pieces of data and a few dollars in call fees. But it is the only way to replace the missing official data.

10. My judgment

String these together, the conclusion is actually quite clear.

There are three things that the architecture gives it, and there is no doubt about it: it is fast, cheap, and it will not answer the wrong questions. These three things are directly determined by the design of "no writing". I can push it out from the outside and will definitely be established without anyone's endorsement.

The core selling point of its promotion has nothing to do with the structure. "If you say you are 80% sure, you will be 80% accurate." This matter is a matter of training. The structure cannot provide any guarantee, and the official has not given any data. In principle, the evaluation method cannot be considered.

What can be proved is sufficient, but there is not a single number to prove.

But I don't want to write this article as a bad-mouthing, because I agree with the judgment it makes:

The vast majority of AI calls in software don't need it to speak at all. Who to assign this work order, whether to block this content, and where to map this field is all a judgment. Our current standard practice is to let a model trained to write first write a paragraph of words, and then use regular rules to extract the words from it as a decision.

The link itself is crooked. We've been paying for "it has to speak first."

Now someone is betting $40 million that this section should be torn down. I think this bet is right. The company's name comes from Jevons, the economist who discovered that "when things become cheaper, total consumption skyrocketed." The meaning was clear: they did not intend to make money on unit prices, but were betting that after machines judged it too cheap to price, developers would Stuff AI into every corner of the software.

!

Finally, a coincidence.

I just wrote a set of studies two days ago about how unreliable it is to use large models to check on AI's work: it is reading tone, and tone is the easiest thing to fake, and what's even more troublesome isthat it never tells you when it is guessing.

Jev grows on this wound: it does not generate tone, only gives judgments, and each judgment is attached to a confidence value.

The direction is right. It's just that the current position is: we have gone from "it doesn't tell you it's guessing" to "it tells you how sure it is, but you can't verify whether this number is accurate."

This is a step forward. But there's only one step.

A few sentences

Jev's pricing, speed multiples, type definition, 255 upper limit, evaluation method and company's self-retention are all from official blogs and public interface documents, and the citations have been marked as official.

The structure is not officially disclosed. The inferences in the article are based on reverse billing evidence provided by independent technical analysis and publicly observable behaviors. The author of the analysis himself also stated that the overall reconstruction has speculative elements and cannot prove the existence of independent classification modules. I handle it according to the same standard: I write what can be excluded from the evidence, and mark what cannot be determined.

I calculated the cost comparison table and the threshold account myself. Assuming they are written in the main text, I can check the calculations myself. The length of the token varies greatly from task to task, the numbers change, and the direction of the conclusion remains unchanged.

Regarding post-training impairment calibration, it is taken from the GPT-4 technical report and measured on the MMLU subset.

Jev is currently queuing for early access. I have not received the actual call permission.No data in the article comes from my own actual measurement. All specific figures are indicated with the source. The verification method in section 9 is the operational advice I gave and has not yet been run on real data.