A.1. Derivation of Variance Calculation Formulas

Mon, 01 Jan 0001 00:00:00 +0000

A.1.1. Derivation of the Youngs and Cramer Method

The Youngs and Cramer method is an enhancement of the Welfold’s online method. To understand the derivation of the Youngs and Cramer method, the Welford method is examined first.

Welfold method:

Welford’s method for calculating variance is defined by the following recurrence relation:

$$ V_{n} = V_{n-1} + \frac{n-1}{n} (x_{n} - A_{n-1})^{2} $$

The derivation of this relation is as follows:

$$ \begin{align*} V_{n} &= \sum_{i=1}^{n} (x_{i} - A_{n})^{2} = \sum_{i=1}^{n-1} (x_{i} - A_{n})^{2} + (x_{n} - A_{n})^{2} \\ &= \sum_{i=1}^{n-1} \left( (x_{i} - \frac{1}{n}((n-1)A_{n-1} + x_{n}) \right)^{2} + \left( (x_{n} - \frac{1}{n}((n-1)A_{n-1} + x_{n}) \right)^{2} \\ &= \sum_{i=1}^{n-1} \left( (x_{i} - A_{n-1}) - \frac{1}{n}(x_{n} - A_{n-1}) \right)^{2} + \left( \frac{1}{n} (nx_{n} - (n-1)A_{n-1} - x_{n} ) \right)^{2} \\ &= \sum_{i=1}^{n-1} \left( (x_{i} - A_{n-1})^{2} - \frac{2(x_{n} - A_{n-1})}{n}(x_{i} - A_{n-1}) + \frac{1}{n^{2}}(x_{n} - A_{n-1})^{2} \right) \\ & \quad + \left( \frac{(n - 1) x_{n} - (n-1) A_{n-1}}{n} \right)^{2} \\ &= \sum_{i=1}^{n-1} (x_{i} - A_{n-1})^{2} - \frac{2(x_{n} - A_{n-1})}{n} \sum_{i=1}^{n-1}(x_{i} - A_{n-1}) + \frac{1}{n^{2}} \sum_{i=1}^{n-1} (x_{n} - A_{n-1})^{2} \\ & \quad + \left( \frac{(n - 1) (x_{n} - A_{n-1})}{n} \right)^{2} \\ &= V_{n-1} - \frac{2(x_{n} - A_{n-1})}{n} \left( \sum_{i=1}^{n-1} x_{i} - (n-1)A_{n-1} \right) + \frac{1}{n^{2}} (n-1) \cdot (x_{n} - A_{n-1})^{2} \\ & \quad + \left( \frac{n-1}{n} \right)^{2} (x_{n} - A_{n-1})^{2} \\ &= V_{n-1} - \frac{2(x_{n} - A_{n-1})}{n} (S_{n-1} - S_{n-1}) + \left( \frac{n-1}{n^{2}} + \left( \frac{n-1}{n}\right)^{2} \right) (x_{n} - A_{n-1})^{2} \\ &= V_{n-1} - \frac{2(x_{n} - A_{n-1})}{n} \cdot 0 + \frac{(n-1) + (n-1)^{2}}{n^{2}} (x_{n} - A_{n-1})^{2} \\ &= V_{n-1} + \frac{(n-1)(1 + (n-1))}{n^{2}} (x_{n} - A_{n-1})^{2} \\ &= V_{n-1} + \frac{n-1}{n} (x_{n} - A_{n-1})^{2} \end{align*} $$

Youngs and Cramer method:

The Youngs and Cramer method modifies Welford’s method by replacing the average $A_{n-1}$ with the sum $S_{n-1}$ to improve computational efficiency and numerical stability.

A.2. io_uring Examples

Mon, 01 Jan 0001 00:00:00 +0000

Knowledge of io_uring implementation examples clarifies Asynchronous I/O (AIO) in PostgreSQL. Below are two simple examples.

These serve as a toy model of a buffer manager, asynchronously reading data from a file into slots in memory (Figure A3.1).

Figure A3.1. Toy Buffer Manager Model.

The programs asynchronously read a 32-byte file (rel.data) into the BufferPool array using four 8-byte read requests.

$ cat rel.data
A0000000B0000001C0000010D0000011

#define BUFFER_SIZE 8
#define PAGE_SIZE 8

char BufferPool[BUFFER_SIZE][PAGE_SIZE + 1];