In short, the purpose of a good calibration kit is to allow you to make useful measurements with a VNA. Here we will explain why a calibration kit is needed. The method of presentation is as follows:
No measuring instruments are perfect, but let us assume hypothetically that some company such as Keysight, Anritsu, Rohde & Schwarz or Copper Mountain was able to create a perfect 6 GHz laboratory vector network analyzer (VNA). They decide to fit a 50 Ohm female N connector on this. This is an unbalanced connector, and typical of the type fitted to lower frequency network analyzers.
Now assume that a customer designs a Yagi for 144 MHz. The dipole element of a Yagi is balanced, and the dipole unbalanced, a balanced to unbalanced transformer (balun) is fitted with a male N connector on it. The customer wishes to measure the impedance of the antenna and balun combination by injecting a signal into the balun, and measuring the amplitude and phase of the reflected signal. In principle, the balun could be screwed directly onto the vector network analyzer, as one has a male N connector and the other a female N connector. However, the close proximity of the antenna to the instrument would totally mess up the measurement of impedance, as the electromagnetic fields generated by the antenna would be reflected from the instrument, back to the antenna. This reflection would mean the instrument is no longer measuring the antenna in free space, but close to a laboratory instrument.
To mitigate this effect, the engineer could in principle add a piece of ideal lossless 50 Ohm cable made from a super conducting material to separate the antenna from the network analyzer. He/she carefully runs the cable at right angles to the elements, and again measures the impedance. Unfortunately, whilst this has solved one problem, it has created another. The electromagnetic radiation does not get from one end of the cable to the other instantly. The delay causes a phase shift, which means that the phase angles are measured completely wrong. So the engineer has no idea of the true impedance at the antenna. Both the real (R) and imaginary (j X) components of the impedance would be totally wrong. We can see that even with a perfect network analyzer, and perfect cable, the impedance measurements would be meaningless.
With this ideal setup, the standard wave ratio (SWR) would be accurately measured, but if you just want to measure SWR, there are cheaper ways than using a vector network analyzer. In any case, even for SWR measurements, a VNA will be seen to be much more accurate than an SWR meter.
Now if we consider that a practical scenario, rather than an idealised one, we will see there are many more sources of error. Some of these are random in nature (e.g. noise, connector repeatability) and can not be corrected by corrected by calibration, but there exists other sources of error which can be categorized into one of 3 categories:
It can be shown that these errors can be corrected mathematically if 3 different and accurately known standards are measured. In principle, using resistors of 1 Ohm, 1.0000001 Ohms and 1.00002 Ohms would work, but in practice it is best to use 3 standards that are as different as possible. These devices are typically a short, open, and 50 Ohm load. So the process of using a VNA is
Since idealised shorts and opens can't be made, and the standards must be accurately defined for the error terms to be accurately computed, a lot of the work in a calibration kit is characterizing the open and short accurately.