Package 'censobr'

Title: Download Data from Brazil's Population Census
Description: Easy access to data from Brazil's population censuses. The package provides a simple and efficient way to download and read the data sets and the documentation of all the population censuses taken in and after 1960 in the country. The package is built on top of the 'Arrow' platform <https://arrow.apache.org/docs/r/>, which allows users to work with larger-than-memory census data using 'dplyr' familiar functions. <https://arrow.apache.org/docs/r/articles/arrow.html#analyzing-arrow-data-with-dplyr>.
Authors: Rafael H. M. Pereira [aut, cre] (ORCID: <https://orcid.org/0000-0003-2125-7465>), Rogério J. Barbosa [aut] (ORCID: <https://orcid.org/0000-0002-6796-4547>), Pedro Herculano Souza [ctb], Diego Rabatone Oliveira [ctb], Neal Richardson [ctb], Haydee Svab [ctb], Ipea - Institute for Applied Economic Research [cph, fnd]
Maintainer: Rafael H. M. Pereira <[email protected]>
License: MIT + file LICENSE
Version: 1.0.0
Built: 2026-09-21 17:23:47 UTC
Source: https://github.com/ipea/censobr

Help Index

Manage cached files from the censobr package

Description

Manage cached files from the censobr package

Usage

censobr_cache(
  list_files = TRUE,
  print_tree = FALSE,
  delete_file = NULL,
  verbose = TRUE
)

Arguments

list_files

Logical. Whether to print a message with the address of all censobr data sets cached locally. Defaults to TRUE.
print_tree

Logical. Whether the cache files should be printed in a tree-like format. This parameter only works if list_files = TRUE. Defaults to FALSE.
delete_file

String. The file name or a string pattern that matches the file path of a file cached locally and which should be deleted. Defaults to NULL, so that no file is deleted. Two values are read as keywords rather than as patterns: delete_file = "all" deletes all of the cached files, and delete_file = "old" deletes only the files cached from previous data releases.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Details

censobr caches data in a directory versioned by data release, so a new data release does not read files downloaded from the previous one. Files from previous releases are deleted automatically the first time data is downloaded in a session, and can be deleted at any time with delete_file = "old". Set options(censobr.keep_old_cache = TRUE) to keep them, for example to go on working with an older data release. Microdata imported with import_microdata22() are never deleted automatically, because censobr cannot download them again.

Value

A message indicating which file exist and/or which ones have been deleted from the local cache directory.

See Also

Other Cache data: get_censobr_cache_dir(), set_censobr_cache_dir()

Examples

# list all files cached
censobr_cache(list_files = TRUE)

# delete particular file
censobr_cache(delete_file = '2010_deaths')

Data dictionary of Brazil's census data

Description

Open on a browser the data dictionary of Brazil's census data.

Usage

data_dictionary(
  year,
  dataset,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
dataset

Character (case insensitive). The type of data dictionary to be opened, either "microdata" or "tracts". With "microdata", the function opens a single Excel file with the data dictionary of all variables of the microdata, available for every census since 1960. With "tracts", it opens the data dictionary of the census tract-level aggregate data, available since 1970.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

Returns the path to the downloaded file. When verbose = TRUE and the session is interactive, the file is also opened and the path is returned invisibly.

See Also

Other Census documentation: interview_manual()

Examples

# Open data dictionary
data_dictionary(
  year = 2022,
  dataset = 'microdata'
  )

data_dictionary(
  year = 2022,
  dataset = 'tracts'
  )

Get path to cache directory for censobr files

Description

Get the path to the cache directory currently being used for for the censobr files

Usage

get_censobr_cache_dir()

Value

Path to cache dir

See Also

Other Cache data: censobr_cache(), set_censobr_cache_dir()

Examples

# get path to cache directory
get_censobr_cache_dir()

Import controlled-access microdata of the 2022 census into censobr cache

Description

Import controlled-access microdata of the sample component of Brazil's 2022 Population Census ("dados controlados") from a zip file obtained from IBGE, convert the csv files it contains into Parquet, and save them in the local censobr cache directory.

Unlike the microdata of previous censuses, the 2022 controlled-access microdata are distributed by IBGE under controlled access and cannot be redistributed by censobr directly. Users need to request the data directly from IBGE at https://microdados.ibge.gov.br/ and run this function once on the zip file they receive from IBGE. From then on, the data are available locally like any other data set cached by the censobr package, and no download is attempted.

One Parquet file is written per table, following the file naming convention used by censobr:

Table in the zip file File written to the cache
Domicilios ⁠2022_households.controlado_<data release>.parquet⁠
Familia ⁠2022_families.controlado_<data release>.parquet⁠
Mortalidade ⁠2022_mortality.controlado_<data release>.parquet⁠
Pessoas ⁠2022_population.controlado_<data release>.parquet⁠

After that the tables sit in the cache and are read from disk, with no further processing. The data imported via import_microdata22() lives in the cache directory even if censobr updates to a new data release. This means you only need to import the data once. Nonetheless, we strongly recommend you store the original .zip from IBGE somewhere safe in case you need to import that data again.

Usage

import_microdata22(zip_path, verbose = TRUE)

Arguments

zip_path

String. Path to the local zip file with the controlled-microdata of the 2022 census sample saved, as provided by IBGE. The original file is expected to hold one subdirectory per state, each containing the csv files of the Domicilios, Familia, Mortalidade and Pessoas tables.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

Returns NULL invisibly and prints message. The function is called for its side effect of writing Parquet files to the censobr cache directory, whose location can be checked with get_censobr_cache_dir().

Examples

## Not run: 

# **pointing to a temp cache dir just so this example does not overite cache**
temp_cache_dir <- fs::path_temp("temp_example")
censobr::set_censobr_cache_dir(temp_cache_dir)

# path to fake zip file
# **the real zip file has to be requested from IBGE beforehand**
path_to_zip <- system.file(
  "extdata/microdata_2022_controlado_fake.zip",
  package = "censobr"
)

# import controlled-access microdata 2022
censobr::import_microdata22(zip_path = path_to_zip)

# check files in cache dir
censobr::censobr_cache()

# set cache back to original dir
censobr::set_censobr_cache_dir(path = NULL)

## End(Not run)

Interview manual of the data collection of Brazil's censuses

Description

Open on a browser the interview manual of the data collection of Brazil's censuses

Usage

interview_manual(year, showProgress = TRUE, cache = TRUE, verbose = TRUE)

Arguments

year

Numeric. Year of reference in the format yyyy.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

Returns the path to the downloaded file. When verbose = TRUE and the session is interactive, the file is also opened and the path is returned invisibly.

See Also

Other Census documentation: data_dictionary()

Examples

# Open interview manual on the browser
interview_manual(
  year = 2010,
  showProgress = FALSE
  )

Questionnaires used in the data collection of Brazil's censuses

Description

Open on a browser the questionnaire used in the data collection of Brazil's censuses

Usage

questionnaire(year, type, showProgress = TRUE, cache = TRUE, verbose = TRUE)

Arguments

year

Numeric. Year of reference in the format yyyy.
type

Character. The type of questionnaire used in the survey, whether the "long" one used in the sample component of the census, or the "short" one, which is answered by more households. Options include c("long", "short").
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

Returns the path to the downloaded file. When verbose = TRUE and the session is interactive, the file is also opened and the path is returned invisibly.

Examples

library(censobr)

# Open questionnaire on browser
questionnaire(year = 2010, type = 'long', showProgress = FALSE)

Download microdata of emigration records from Brazil's census

Description

Download microdata of emigration records from Brazil's census. Data collected in the sample component of the questionnaire.

Usage

read_emigration(
  year,
  columns = NULL,
  add_labels = NULL,
  merge_households = FALSE,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
columns

String. A vector of column names to keep. The rest of the columns are not read. Defaults to NULL and read all columns. If the user passes merge_households = TRUE, then the columns vector can also include the names of the households data set that should be kept.
add_labels

Character. Whether the function should add labels to the responses of categorical variables. When add_labels = "pt", the function adds labels in Portuguese. Defaults to NULL.
merge_households

Logical. Indicate whether the function should merge household variables to the output data. Defaults to FALSE.
as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

An arrow Dataset or a "data.frame" object.

See Also

Other Microdata: read_families(), read_households(), read_mortality(), read_population()

Examples

# return data as arrow Dataset
df <- read_emigration(
  year = 2010,
  showProgress = FALSE
  )

# return data as data.frame
df <- read_emigration(
  year = 2010,
  as_data_frame = TRUE,
  showProgress = FALSE
  )

Download microdata of family records from Brazil's census

Description

Download microdata of family records from Brazil's census. Data collected in the sample component of the questionnaire.

Usage

read_families(
  year,
  columns = NULL,
  add_labels = NULL,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
columns

String. A vector of column names to keep. The rest of the columns are not read. Defaults to NULL and read all columns. If the user passes merge_households = TRUE, then the columns vector can also include the names of the households data set that should be kept.
add_labels

Character. Whether the function should add labels to the responses of categorical variables. When add_labels = "pt", the function adds labels in Portuguese. Defaults to NULL.
as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

An arrow Dataset or a "data.frame" object.

See Also

Other Microdata: read_emigration(), read_households(), read_mortality(), read_population()

Examples

# return data as arrow Dataset
df <- read_families(
  year = 2000,
  showProgress = FALSE
  )

Download microdata of household records from Brazil's census

Description

Download microdata of household records from Brazil's census. Data collected in the sample component of the questionnaire.

Usage

read_households(
  year,
  columns = NULL,
  add_labels = NULL,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
columns

String. A vector of column names to keep. The rest of the columns are not read. Defaults to NULL and read all columns. If the user passes merge_households = TRUE, then the columns vector can also include the names of the households data set that should be kept.
add_labels

Character. Whether the function should add labels to the responses of categorical variables. When add_labels = "pt", the function adds labels in Portuguese. Defaults to NULL.
as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

An arrow Dataset or a "data.frame" object.

1960 Census

The 1960 microdata version available in censobr is a combination of two versions of the Demographic Census sample. The 25% sample data from the 1960 Census was never fully processed by IBGE - several states did not have their questionnaires digitized. Currently, this dataset only has data from 16 states of the Federation (and from a contested border region between Minas Gerais and Espirito Santo called Serra dos Aimores). Information is missing for the states of the former Northern Region, Maranhão, Piaui, Guanabara, Santa Catarina, and Espírito Santo. In 1965, IBGE decided to draw a probabilistic sub-sample of approximately 1.27% of the population, including all units of the federation. With this data, IBGE produced several official reports at the time. The data from censobr is the combination of these two datasets.

We pre-processed the 1.27% sample data to ensured data consistency, given the original data was partially corrupted. We also created a sample weight variable to correct for unbalanced data and to expand te sample to the total population. For the data from the 25% sample, the weights expand to the municipal totals. Meanwhile, for the data from the 1.27% sample, the weights expand to the state totals. Additionally, we constructed a few variables that allow for the approximate incorporation of the complex sample design, enabling the proper calculation of standard errors and confidence intervals.

You can read more about the 1960 Census and find a thorough documentation of how this dataset was processed on this link https://github.com/antrologos/ConsistenciaCenso1960Br.

See Also

Other Microdata: read_emigration(), read_families(), read_mortality(), read_population()

Examples

# return data as arrow Dataset
df <- read_households(
  year = 2010,
  showProgress = FALSE
  )

Download microdata of death records from Brazil's census

Description

Download microdata of death records from Brazil's census. Data collected in the sample component of the questionnaire.

Usage

read_mortality(
  year,
  columns = NULL,
  add_labels = NULL,
  merge_households = FALSE,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
columns

String. A vector of column names to keep. The rest of the columns are not read. Defaults to NULL and read all columns. If the user passes merge_households = TRUE, then the columns vector can also include the names of the households data set that should be kept.
add_labels

Character. Whether the function should add labels to the responses of categorical variables. When add_labels = "pt", the function adds labels in Portuguese. Defaults to NULL.
merge_households

Logical. Indicate whether the function should merge household variables to the output data. Defaults to FALSE.
as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

An arrow Dataset or a "data.frame" object.

See Also

Other Microdata: read_emigration(), read_families(), read_households(), read_population()

Examples

# return data as arrow Dataset
df <- read_mortality(
  year = 2010,
  showProgress = FALSE
  )

# return data as data.frame
df <- read_mortality(
  year = 2010,
  as_data_frame = TRUE,
  showProgress = FALSE
  )

Download microdata of population records from Brazil's census

Description

Download microdata of population records from Brazil's census. Data collected in the sample component of the questionnaire.

Usage

read_population(
  year,
  columns = NULL,
  add_labels = NULL,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE,
  merge_households = FALSE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
columns

String. A vector of column names to keep. The rest of the columns are not read. Defaults to NULL and read all columns. If the user passes merge_households = TRUE, then the columns vector can also include the names of the households data set that should be kept.
add_labels

Character. Whether the function should add labels to the responses of categorical variables. When add_labels = "pt", the function adds labels in Portuguese. Defaults to NULL.
as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.
merge_households

Logical. Indicate whether the function should merge household variables to the output data. Defaults to FALSE. When merge_households = TRUE, it is mandatory to pass the columns argument to select which columns should be kept. See the Details section.

Details

merge_households = TRUE is available for every census year the function serves, and requires columns to be set. For 1980 and 1991 the population microdata already include all variables of the household data set, so merge_households = TRUE has no effect and a message says so. Merging household variables into the full population microdata produces about 300 columns and can require more than 20GB of memory; naming the columns you need keeps the operation fast and light, typically a few seconds. The merge writes a temporary parquet file that is removed when the R session ends.

Value

An arrow Dataset or a "data.frame" object.

1960 Census

The 1960 microdata version available in censobr is a combination of two versions of the Demographic Census sample. The 25% sample data from the 1960 Census was never fully processed by IBGE - several states did not have their questionnaires digitized. Currently, this dataset only has data from 16 states of the Federation (and from a contested border region between Minas Gerais and Espirito Santo called Serra dos Aimores). Information is missing for the states of the former Northern Region, Maranhão, Piaui, Guanabara, Santa Catarina, and Espírito Santo. In 1965, IBGE decided to draw a probabilistic sub-sample of approximately 1.27% of the population, including all units of the federation. With this data, IBGE produced several official reports at the time. The data from censobr is the combination of these two datasets.

We pre-processed the 1.27% sample data to ensured data consistency, given the original data was partially corrupted. We also created a sample weight variable to correct for unbalanced data and to expand te sample to the total population. For the data from the 25% sample, the weights expand to the municipal totals. Meanwhile, for the data from the 1.27% sample, the weights expand to the state totals. Additionally, we constructed a few variables that allow for the approximate incorporation of the complex sample design, enabling the proper calculation of standard errors and confidence intervals.

You can read more about the 1960 Census and find a thorough documentation of how this dataset was processed on this link https://github.com/antrologos/ConsistenciaCenso1960Br.

See Also

Other Microdata: read_emigration(), read_families(), read_households(), read_mortality()

Examples

# return data as arrow Dataset
df <- read_population(
  year = 2010,
  showProgress = FALSE
  )

Download census tract-level data from Brazil's censuses

Description

Download census tract-level aggregate data from Brazil's censuses.

Usage

read_tracts(
  year,
  dataset,
  as_data_frame = FALSE,
  showProgress = TRUE,
  cache = TRUE,
  verbose = TRUE
)

Arguments

year

Numeric. Year of reference in the format yyyy.
dataset

Character. The dataset to be opened. The following options are available for each edition of the census:

2000 Census

  • c("Basico", "Domicilio", "Responsavel", "Pessoa", "Instrucao", "Morador").

2010 Census

  • c("Basico", "Domicilio", "DomicilioRenda", "Responsavel", "ResponsavelRenda", "Pessoa", "PessoaRenda", "Entorno").

2022 Census

  • c("Basico", "Domicilio", "ResponsavelRenda", "Pessoas", "Indigenas", "Quilombolas", "Entorno", "Obitos", "Preliminares").

The censobr package exposes all original IBGE census tracts datasets, regrouping them into broader themes and appending geographic identifiers so that they align seamlessly with geobr shapefiles.

For a complete description of the datasets, themes, and variables, check

  • data_dictionary(year = 2000, dataset = "tracts"),

  • data_dictionary(year = 2010, dataset = "tracts"),

  • data_dictionary(year = 2022, dataset = "tracts").

as_data_frame

Logical. When FALSE (Default), the function returns an Arrow Dataset, which allows users to work with larger-than-memory data. If TRUE, the function returns data.frame.
showProgress

Logical. Defaults to TRUE display download progress bar. The progress bar only reflects only the downloading time, not the time to load the data to memory.
cache

Logical. Whether the function should read the data cached locally, which is much faster. Defaults to TRUE. The first time the user runs the function, censobr will download the file and store it locally so that the file only needs to be download once. If FALSE, the function will download the data again and overwrite the local file.
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

An arrow Dataset or a "data.frame" object.

Examples

library(censobr)

# return data as arrow Dataset
df <- read_tracts(
  year = 2022,
  dataset = 'Domicilio',
  showProgress = FALSE
  )

# return data as data.frame
df <- read_tracts(
  year = 2010,
  dataset = 'Basico',
  as_data_frame = TRUE,
  showProgress = FALSE
  )

Set custom cache directory for censobr files

Description

Set custom directory for caching files from the censobr package. The user only needs to run this function once. This set directory is persistent across R sessions.

Usage

set_censobr_cache_dir(path, verbose = TRUE)

Arguments

path

String. The path to an existing directory. It defaults to path = NULL, to use the default directory
verbose

A logical. Whether the function should print informative messages. Defaults to TRUE.

Value

A message pointing to the directory where censobr files are cached.

See Also

Other Cache data: censobr_cache(), get_censobr_cache_dir()

Examples

# Set custom cache directory
tempd <- tempdir()
set_censobr_cache_dir(path = tempd)

# back to default path
set_censobr_cache_dir(path = NULL)