Editing Consumer price index (section)

==Weighting==

===Weights and sub-indices===
By convention, weights are fractions or ratios summing to one, as percentages summing to 100 or as per mile numbers summing to 1000.{{Citation needed|reason=cite examples, especially regarding mille numbers?  Are any other totals used? |date=April 2013}}

On the European Union's Harmonized Index of Consumer Prices (HICP), for example, each country computes some 80 prescribed sub-indices, their weighted average constituting the national HICP. The weights for these sub-indices will consist of the sum of the weights of a number of component lower level indices. The classification is according to use, developed in a national accounting context. This is not necessarily the kind of classification that is most appropriate for a consumer price index. Grouping together of substitutes or of products whose prices tend to move in parallel might be more suitable.

For some of these lower-level indices detailed reweighting to make them be available,{{clarify|date=July 2020}} allowing computations where the individual price observations can all be weighted.{{clarify|date=July 2020}} This may be the case, for example, where all selling is in the hands of a single national organization which makes its data available to the index compilers. For most lower level indices, however, the weight will consist of the sum of the weights of a number of elementary aggregate indices, each weight corresponding to its fraction of the total annual expenditure covered by the index. An 'elementary aggregate' is a lowest-level component of expenditure: this has a weight, but the weights of each of its sub-components are usually lacking. Thus, for example: Weighted averages of elementary aggregate indices (e.g. for men's shirts, raincoats, women's dresses, etc.) make up low-level indices (e.g. outer garments).

Weight averages of these, in turn, provide sub-indices at a higher, more aggregated level (e.g. clothing) and weighted averages of the latter provide yet more aggregated [[sub-indices]] (e.g. Clothing and Footwear).

Some of the elementary aggregate indices and some of the sub-indices can be defined simply in terms of the types of goods and/or services they cover. In the case of such products as newspapers in some countries and postal services, which have nationally uniform prices.{{clarify|date=July 2020}}{{Fix|text=words missing?}} But where price movements do differ or might differ between regions or between outlet types, separate regional and/or outlet-type elementary aggregates are ideally required for each detailed category of goods and services, each with its own weight. An example might be an elementary aggregate for sliced bread sold in supermarkets in the Northern region.

Most elementary aggregate indices are necessarily 'unweighted' averages for the sample of products within the sampled outlets. However, in cases where it is possible to select the sample of outlets from which prices are collected so as to reflect the shares of sales to consumers of the different outlet types covered, self-weighted elementary aggregate indices may be computed. Similarly, if the market shares of the different types of products represented by product types are known, even only approximately, the number of observed products to be priced for each of them can be made proportional to those shares.

===Estimating weights===
The outlet and regional dimensions noted above mean that the estimation of weights involves a lot more than just the breakdown of expenditure by types of goods and services, and the number of separately weighted indices composing the overall index depends upon two factors:
# The degree of detail to which available data permit breakdown of total consumption expenditure in the weight reference-period by type of expenditure, region and outlet type.
# Whether there is reason to believe that price movements vary between these most detailed categories.

How the weights are calculated, and in how much detail, depends upon the availability of information and upon the scope of the index. In the UK the retail price index (RPI)<ref>{{Cite web|url=https://www.investopedia.com/terms/r/rpi.asp|title = Retail Price Index (RPI) Definition}}</ref> does not relate to the whole of consumption, for the reference population is all private households with the exception of pensioner households that derive at least three-quarters of their total income from state pensions and benefits, and "high income households" whose total household income lies within the top four per cent of all households. The result is that it is difficult to use data sources relating to total consumption by all population groups.

For products whose price movements can differ between regions and between different types of outlet:
* The ideal, rarely realizable in practice, would consist of estimates of expenditure for each detailed consumption category, for each type of outlet, for each region.
* At the opposite extreme, with no regional data on expenditure totals but only on population (e.g. 24% in the Northern region) and only national estimates for the shares of different outlet types for broad categories of consumption (e.g. 70% of food sold in supermarkets) the weight for sliced bread sold in supermarkets in the Northern region has to be estimated as the share of sliced bread in total consumption × 0.24 × 0.7.

The situation in most countries comes somewhere between these two extremes. The point is to make the best use of whatever data are available.

Due to differences in weightings in the consumer basket, different price indices may be calculated for groups with various demographic characteristics. For example, consumer price indices calculated according to the weightings in the consumer basket of income groups may show significantly different trends.<ref>{{Cite journal |last=Daşdemir |first=Esat |date=2022-04-15 |title=A New Proposal for Consumer Price Index (CPI) Calculation and Income Distribution Measurement by Income Groups |url=https://iupress.istanbul.edu.tr/en/journal/jecs/article/gelir-gruplarina-gore-tuketici-fiyat-endeksi-tufe-hesaplari-ve-gelir-dagilimi-olcumu-icin-yeni-bir-oneri |journal=Journal of Economy Culture and Society |language=en |volume=0 |issue=65 |pages=395-414 |doi=10.26650/JECS2021-984480 |issn=2602-2656}}</ref>

===The nature of the data used for weighting===
No firm rules can be suggested on this issue for the simple reason that the available statistical sources differ between countries. However, all countries conduct periodical household-expenditure surveys and all produce breakdowns of consumption expenditure in their [[national accounts]]. The expenditure classifications used there may however be different. In particular:
* Household-expenditure surveys do not cover the expenditures of foreign visitors, though these may be within the scope of a consumer price index.
* National accounts include [[imputed rent]]s for owner-occupied dwellings which may not be within the scope of a consumer price index.

Even with the necessary adjustments, the national-account estimates and household-expenditure surveys usually diverge.

The ''statistical sources'' required for regional and outlet-type breakdowns are usually weak. Only a large-sample Household Expenditure survey can provide a regional breakdown. Regional population data are sometimes used for this purpose, but need adjustment to allow for regional differences in living standards and consumption patterns. Statistics of retail sales and market research reports can provide information for estimating outlet-type breakdowns, but the classifications they use rarely correspond to COICOP categories.

The increasingly widespread use of bar-code scanners in shops has meant that detailed cash register printed receipts are provided by shops for an increasing share of retail purchases. This development makes possible improved Household Expenditure surveys, as Statistics Iceland has demonstrated. Survey respondents keeping a diary of their purchases need to record only the total of purchases when itemized receipts were given to them and keep these receipts in a special pocket in the diary. These receipts provide not only a detailed breakdown of purchases but also the name of the outlet. Thus response burden is markedly reduced, accuracy is increased, product description is more specific and point of purchase data are obtained, facilitating the estimation of outlet-type weights.

There are only two general principles for the estimation of weights: use all the available information and accept that rough estimates are better than no estimates.

===Reweighting===
Ideally, in computing an index, the weights would represent current annual expenditure patterns. In practice, they necessarily reflect past using the most recent data available or, if they are not of high quality, some average of the data for more than one previous year. Some countries have used a three-year average in recognition of the fact that household survey estimates are of poor quality. In some cases, some of the data sources used may not be available annually, in which case some of the weights for lower-level aggregates within higher-level aggregates are based on older data than the higher level weights.

Infrequent reweighting saves costs for the national statistical office but delays the introduction into the index of new types of expenditure. For example, subscriptions for Internet service entered index compilation with a considerable time lag in some countries, and account could be taken of digital camera prices between re-weightings only by including some digital cameras in the same elementary aggregate as film cameras.