Convert probability to odds
For a binary event with probability strictly between zero and one, odds are p divided by one minus p; logit is the natural logarithm of those odds. Probability 0.8 gives odds four and logit about 1.386. Probability 0.2 gives odds one quarter and logit about minus 1.386. A difference in probabilities is not a difference on the logit scale. Zero logit corresponds to probability one half, and a negative logit is perfectly valid. Convert percentages to fractions before calculation: eighty percent means 0.8, not the number eighty.
Invert with the sigmoid
The sigmoid maps a real logit z back to probability: one divided by one plus exp of minus z. It gives 0.5 at zero and 0.8 at log four. Mathematically finite inputs produce values strictly between zero and one, but floating-point outputs can round to endpoints, and naive exponential evaluation can overflow. Use a numerically stable implementation. Clipping endpoint probabilities before taking logarithms changes the calculation and should be reported. It is also distinct from selecting a decision threshold, which turns a score into an action rather than changing its numerical representation.
Read two-class softmax scores
Neural-network outputs before normalization are also commonly called logits. For two class scores(z0,z1), softmax probability for class one equals sigmoid of z1 minus z0. Scores(0,log four) and(10,10 plus log four) both give 0.8. Adding the same constant to every score leaves softmax probabilities unchanged. The absolute value of one raw score therefore cannot be interpreted alone as probability or confidence. Identify the reference class and normalization rule, especially when comparing saved scores across different implementations or inspecting a model’s final linear layer.
Normalize a multiclass vector
Multiclass softmax exponentiates each score and divides by the sum of exponentials. Subtracting the maximum score from every entry preserves probabilities while avoiding large positive exponentials. Check nonnegativity and a total of one. Successful normalization does not establish correctness or calibration: a model can be confidently wrong. Changing the candidate class set changes the denominator as well. Consequently, probabilities produced under different candidate sets are not automatically comparable as if only a single score had changed. Keep the class vocabulary and normalization context with stored model outputs.
Separate temperature and calibration
Dividing all scores by one positive temperature T flattens the distribution as T increases and sharpens it as T decreases. Sharing a positive T preserves class ordering and the argmax, although probability-threshold decisions can change. Fitting a temperature on suitable validation data is one calibration method. Arbitrarily choosing a temperature does not guarantee calibrated probabilities or correct predictions. Also distinguish calibration for an evaluation task from changing a language model’s sampling temperature during generation. The same mathematical transformation can participate in different procedures with different intended outcomes.
Check representations before interpreting them
Convert probabilities 0.2,0.5, and 0.8 to odds and logits, then apply sigmoid to recover them. Add a common constant one hundred to the two scores(0,log four) and verify that softmax is unchanged. Completion means distinguishing raw scores, log odds, normalized probabilities, and thresholded class decisions. Whether a reported 0.8 behaves like an eighty-percent frequency is a separate empirical question. After checking the transformations, use the calibration article to investigate that question with observations rather than inferring reliability from the score representation alone.