This report provides information on the quality of the data used to produce the official statistics in the following DfI publications from the CHS Survey:
The report is structured around the five quality dimensions for statistical outputs (from the European Statistics Code of Practice, PDF (458 KB). The UK Statistics Authority’s Code of Practice for Statistics requires that:
6.7 Base methods on national or international good practice, scientific principles or professional consensus. Identify potential bias and address limitations. Use recognised standards, classifications and definitions. Explain reasons for deviations from these standards and any related implications for use.
6.10 Verify that the statistics are representative and of suitable quality and monitor relevant quality dimensions for both input data and the statistics, such as completeness and validity, accuracy and reliability, coherence and comparability, and timeliness. Quantify statistical error, including bias, and produce measures of confidence, where possible.
7.2 Report on the key quality dimensions such as accuracy and timeliness, and, where possible, give estimates of error and confidence for the statistics. Summarise how uncertainty in the estimates may impact use by using qualifying words, numbers or graphics.
The Continuous Household Survey (CHS) is one of the largest continuous surveys carried out in Northern Ireland. The survey is designed, conducted and analysed by the Central Survey Unit (CSU) of the Northern Ireland Statistics and Research Agency (NISRA). It is based on a sample of the general population resident in private households and has been running since 1983. The survey is designed to provide a regular source of information on a wide range of social and economic issues relevant to Northern Ireland.
Usually, DfI includes six or seven modules in the Continuous Household Survey and these modules vary from year to year. As such content can change but regularly placed modules cover topics such as Road Safety, Travel to/from School, Journey Planning and Walking, Household Water Usage, Cycling & Public Transport.
Due to the coronavirus (COVID-19) pandemic, data collection for the 2020/21 survey moved from face-to-face interviewing to telephone mode with a reduction in the number of questions. Telephone interviewing has remained in place for the 2021/22 survey, therefore caution should be used when interpreting trend data due to methodology and content changes during the 2020/21 survey period. In 2020/21, the output from the survey, in terms of the range and depth of topics covered, was reduced from that of previous years, with the subsequent capacity for detailed analysis constrained. In 2021/22, the sample size for the CHS returned to normal and while the survey content was slightly impacted by the move to telephone interviewing, a larger number of questions were asked. From July 2022 onwards, CSU reinstated the use of face-to-face interviewing, or CAPI (Computer Assisted Personal Interviewing). This meant that the survey became dual-modality – as both Telephone and Face-to-Face interviewing were carried out.
Each year CSU requests updates to the content of the questionnaire modules or proposed new modules, in consultation with client departments. The questionnaire consists of both a household interview and an individual interview with each person aged 16 and over. Both the household and individual questionnaires consist of core items that are included each year, modules that recur on a regular cycle and ad hoc modules.
Core items include accommodation, tenure, employment status, employment activity, educational qualifications, adult health and family information.
Non-core items include attitudes to walking and cycling, method of travel to school by children, household water usage, road safety and participation and attendance at sports and leisure activities. For each of these, a population level analysis is included and a breakdown by various sections of society, including Section 75 categories, where possible.
In DfI, modules are reviewed each year to ensure continued relevance to Departmental Policy. This process is managed by statistical staff, who liaise with relevant policy managers to ensure suitable content is developed and that timescales and analysis outputs are understood and useful to policy customers.
The survey is asked of a systematic random representative sample of the Northern Ireland population therefore there are no major coverage issues. The CHS sample is drawn from the NISRA Address Register which contains address information for every property in NI, excluding those people living in institutions.
Information on data processing – As the survey is dual modality, data is collected both by telephone interview using CATI (Computer assisted Telephone Interviewing) and via face-to-face interviewing, or CAPI (Computer Assisted Personal Interviewing). The interviews are spread equally over the 12 months from April to March. Data is returned from the field via FTP dial up. Data is downloaded onto secure network drives and processed through the CHS data management system. Data is coded and fully validated by CSU statistical staff.
Information on quality control and quality assurance – The CHS uses the ONS Primary and Secondary Harmonised questions as far as possible and where appropriate. Questions are pre-tested before they go into field. The CATI questionnaire is scripted in Blaise which allows CSU staff to define range and consistency checks for each question and to control routing throughout the questionnaire.
The data is subject to further validation checks including treatment of outliers and detailed consistency checking including non-credible checks. A 100% check of all coded data is undertaken by statistical staff to ensure that coding procedures are of a high standard. All completed CHS interviews are back-checked with respondents to ensure that interviews are valid. Interviewers undergo intensive induction training and are trained (face-to-face) before working on CHS.
Sample Design and Sampling Frame – The CHS is based on a systematic random sample of 9,000 addresses drawn each year from the NISRA Address Register. The addresses on the NISRA Address Register are sorted by district council and ward, so the sample is effectively stratified geographically. A starting point in the sampling frame is randomly selected and then every nth (fixed interval) is selected by counting through the frame.
To reduce survey length, the questionnaire consisted of a split sample. This means that depending on the serial number of the questionnaire (even or odd) the respondents would be exposed to a different set of questions for part of the questionnaire. DfI modules were included in even versions of the questionnaire only.
Response Rate – In 2024/25, 3,790 households took part in the survey. When ineligible addresses are discounted from the sampling frame this gives a survey response rate of 46%. Within those 3,790 households, a total of 4,584 individuals took part in the survey (1.2 individuals per household).
Non Response Error/Bias – In any survey there is a possibility of non-response bias. Non-response bias arises if the characteristics of non-respondents differ from those of respondents in such a way that they are reflected in the responses given in the survey. Accurate estimates of non-response bias can be obtained by comparing characteristics of the achieved sample with the distribution of the same characteristics in the population at the time of sampling. Such comparisons are usually made to the current Census of Population data. To assess how accurately the Continuous Household Survey sample reflects the population of Northern Ireland CSU compared the sample with characteristics of the Northern Ireland population from Mid-Year Population Estimates. These calculations were used to derive suitable weighting to be used in general analysis to help correct for any non-response bias. Full details of this comparison may be found in the CHS technical report.
DfI carry out checks on each module to determine whether weighting should be applied and each publication will inform the reader whether weights have been applied or not.
Proxy responses – On CHS, there are no proxy interviews. The household schedule is answered by the first adult to be interviewed on behalf of the whole household.
Imputation rates – No imputation of data occurs on CHS.
Sampling Error – Since the CHS data is based on a sample rather than the whole populations it is subject to sampling error. Sampling error is the difference between the estimate derived from a sample and the ‘true’ value that would result if a census of the whole population were taken under the same conditions. For a simple random sample design, in which every member of the sampled population has an equal and independent chance of inclusion in the sample, the sampling error of any percentage, p, can be calculated by the formula:
s.e. (p) = √(p*(100 – p)/n
where n is the number of respondents on which the percentage is based. The sample for the Continuous Household Survey is drawn as a random sample, and thus this formula can be used to calculate the sampling error of any percentage estimate from the survey. A confidence interval for the population percentage can be calculated by the formula
95% confidence interval = p+/- 1.96 * s.e. (p)
If 100 similar, independent samples were chosen from the same population, 95 of them would be expected to yield an estimate for the percentage, p, within this confidence interval. The absence of design effects in the survey, and therefore of the need to calculate complex standard errors, means that standard statistical tests of significance (which assume random sampling) can be applied directly to the data.
The CHS runs on a financial year cycle (Apr-Mar) and the data is updated three months after the close of the data collection cycle. Data is normally provided to module owners in and around the July that follows the completed financial year. For example, the CHS results based on the 2024/25 survey cycle became available in June 2025.
Given available resource in the Branch, ASRB aim to publish all DfI publications by the end of the financial year in which data are received. That is all data received in June 2025 should have been analysed and published by the 31st March 2026. Order of publication is prioritised through liaison with policy customers and through ASRB’s understanding of departmental priorities.
For each module that DfI include on the CHS, a report is published (in line with The Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018) complete with infographics and data tables in both Excel and ODS format and these are published on the ASRB homepage:
https://www.infrastructure-ni.gov.uk/topics/dfi-statistics-and-research
The main report for each module contains a ‘Technical Notes’ section, outlining important information of the survey including data collection methods, an assessment of data quality, respondent information, information on weighting and other associated survey information.
Information from the CHS is published on the CSU website:
https://www.nisra.gov.uk/statistics/find-your-survey/continuous-household-survey
A bespoke information request service is available to all users of CHS. CHS data is deposited annually in the UK Data Archive and is available to any user wishing to carry out individual analysis.
Most modules have been included in the CHS where there is no existing data source and as such there is no exact comparison with any other data sources. However, comparisons are made to ensure the demographic breakdowns (age and sex) are in line with the mid-year estimates of population. Where a similar question is asked in another survey (e.g. car ownership from the Travel Survey) comparisons are made to check that data are consistent. In 2024/25, travel to school results were compared against the Census results and were consistent.
All surveys conducted by Central Survey Unit follow harmonisation guidelines produced by ONS. This means that, especially for demographic data, questions are asked in the same way and are therefore comparable.
Usually, where modules or questions have been asked in previous years of the CHS, the latest data are always compared with previous years’ data and any significant changes are thoroughly checked and validated.
It could be considered that there is some trade-off between ‘relevance’ and ‘accuracy and reliability’. The sample size of the survey means that sometimes limited analysis can take place for specific travel modes, geographic and other sub-population breakdowns.
Continuous Household Survey data are only analysed if the sample size is large enough for robust analysis.
Policy need also means that priorities can drive change in the modules and that leads to a trade-off between ‘relevance’ and ‘comparability’ as content can change year to year depending on what need exists. Time series can stop suddenly due to changing priorities.
Data presented in this report help to meet the information needs of a wide range of internal and external users (see “Relevance” section for more details of users and uses of the CHS reports and data). In addition to the reports, CHS data are used to answer a number of ad-hoc queries throughout the year on a wide variety of topics. Given the model employed to develop modules, user needs are discussed on an annual basis and module content developed to reflect that emerging need and how data already published has been used. Modules are then updated based on learning from those conversations.
ASRB have a mailing list of CHS users for each publication who are notified each time a relevant CHS publication is issued. This list was recently refreshed to ensure it is up to date and that the people on it still wish to receive information about a specific CHS publication, in consideration of GDPR requirements.
There is an on-going ASRB customer satisfaction survey which includes a question on whether ASRB publications meet customers’ needs. The link to the survey is issued every time users are notified that a report has been published or an ad-hoc query is answered. Results are collated on a quarterly basis. User needs are prioritised, taking account of resources available and the relevance and feasibility of any additional needs identified.
An annual survey control return is completed for the Continuous Household Survey. The compliance burden is calculated from the median time taken to complete the survey and the number of respondents.
The achieved response rate is a good indication of respondent burden. It was 46% for the CHS in 2024/25. In general, social surveys have seen a decrease in response rate in recent years.
CHS data received by ASRB are anonymised (name/address information is not contained in the database). In addition, the data represent a sample of the population and there is no way for ASRB to identify the sample that was chosen from the population.
The data are held either on a secure and accredited network drive that is only accessible to ASRB staff or within DfI’s secure record management system (CM), also with restricted access and subject to standard record retention protocols.
Analysis is only carried out if the sample size is large enough and the data are presented as averages or percentages. There are therefore no small cells which need to have disclosure control methods applied. ASRB has produced a statement on its confidentiality processes which can be found in the Department for Infrastructure’s Code of Practice for Statistics supporting statements: