AN1798

Calibration and linearization technique

11/16

Example of results with this software:

The software determine the averages of the two voltage levels (0 and 2.5 V) which are

applied to Channel 0 and Channel 1 respectively and these results are shown in the

debugger:

ADC_Min: 0x963 (0 V) = 2403 decimal (average value)

ADC_Max: 0x66E (2.5 V) = 1646 decimal (average value)

Then, the software determines their unsigned values:

Unsigned_Min (0 V) = 0x800 - ((~0x963 & 0xFFF) + 1)

= 0x165

= 357 decimal

Unsigned_Max (2.5 V) = 0x800 + 0x66E

= 0xE6E

= 3694 decimal

Then the software determines the sweep of conversion:

Sweep = Unsigned_Max - Unsigned_Min

= 0xE6E - 0x165

= 0xD09

= 3337 decimal

Finally it determines the gain G and the quantum q:

q = 2.5V/3337 = 749 µV

For example a voltage level = 1.60 V is applied to Channel 0, we get:

Conversion_Result = 0x124

Unsigned_Result = 0x800 + 0x124 = 0x924

The calibrated conversion result after subtracting the offset is:

Calibrated_Result_Data = Unsigned_Result - Unsigned_Min

= 0x9C0 - 0x165

= 0x85B

= 2139 decimal

It is possible also to determine the applied voltage value in Channel 0 by multiplying this

result by the quantum:

Calibrated_Result_Volt = 2139 * 749 µV = 1602111µV ~ 1.60 V

Ideally 1.60V converted value is: 1334.8 * 1.60 = 2136

The accuracy of conversion is: 2136 / 2139 = 99.8%