istatR: Interface to the Italian National Institute of Statistics ('ISTAT') API

Description

Provides an interface to the 'ISTAT' 'SDMX' RESTful API https://esploradati.istat.it/SDMXWS. Allows users to discover available datasets, explore their structure and dimensions, and retrieve statistical data from the Italian National Institute of Statistics. Based on the Python 'istatapi' package by Jacopo Attolini.

The istatR package provides an R interface to the ISTAT SDMX RESTful API, allowing users to discover available datasets, explore their structure and dimensions, and retrieve statistical data from the Italian National Institute of Statistics.

Main Functions

all_available

List all available ISTAT datasets

search_dataset

Search datasets by keyword

istat_dataset

Create a dataset object for exploration

get_data

Retrieve data from a dataset

Typical Workflow

library(istatR)

# 1. Explore available datasets
datasets <- all_available()
import_ds <- search_dataset("import")

# 2. Create a dataset object
ds <- istat_dataset("139_176")

# 3. Explore dimensions
dimensions_info(ds)
get_dimension_values(ds, "TIPO_DATO")

# 4. Set filters
ds <- set_filters(ds,
  FREQ = "M",
  TIPO_DATO = c("ISAV", "ESAV"),
  PAESE_PARTNER = "WORLD"
)

# 5. Retrieve data
data <- get_data(ds)

Author(s)

Maintainer: Juan Ignacio Fulponi jfulponi@economicas.uba.ar

See Also

Useful links:

• https://github.com/jfulponi/istatR

• Report bugs at https://github.com/jfulponi/istatR/issues

Useful links:

• https://github.com/jfulponi/istatR

• Report bugs at https://github.com/jfulponi/istatR/issues

List all available ISTAT datasets

Description

Retrieves a list of all available datasets (dataflows) from the ISTAT API.

Usage

all_available()

Value

A tibble with the following columns:

df_id

Dataflow ID

version

Dataset version

df_description

English description of the dataset

df_structure_id

Data structure definition ID

Examples

## Not run: 
# Get all available datasets
datasets <- all_available()
head(datasets)

## End(Not run)

ISTAT API Base Functions

Description

Core functions for communicating with the ISTAT SDMX REST API

Get information about dataset dimensions

Description

Returns information about the dimensions of a dataset, including their positions and associated codelists.

Usage

dimensions_info(dataset, include_descriptions = TRUE)

Arguments

Value

A tibble with dimension information

Examples

## Not run: 
ds <- istat_dataset("139_176")
dimensions_info(ds)

## End(Not run)

ISTAT Dataset Discovery Functions

Description

Functions for discovering and exploring ISTAT datasets

Get ISTAT agency ID

Description

Get ISTAT agency ID

Usage

get_agency_id()

Value

Character string with the agency ID

Get all available values for all dimensions

Description

Uses the availableconstraint endpoint to get all valid values for each dimension. This is more accurate than getting values from codelists as it reflects actual data availability.

Usage

get_available_values(dataset)

Arguments

Value

A named list where each element contains a tibble of available values for that dimension

Examples

## Not run: 
ds <- istat_dataset("139_176")
available <- get_available_values(ds)
available$FREQ  # Available frequency values

## End(Not run)

Get ISTAT API base URL

Description

Get ISTAT API base URL

Usage

get_base_url()

Value

Character string with the base URL

Retrieve data from an ISTAT dataset

Description

Fetches data from an ISTAT dataset using the currently set filters. The data is returned as a tibble with the TIME_PERIOD column converted to Date format and sorted in ascending order.

Usage

get_data(
  dataset,
  start_period = NULL,
  end_period = NULL,
  last_n_observations = NULL
)

Arguments

Value

A tibble containing the requested data with columns including:

DATAFLOW

Dataset identifier

FREQ

Frequency

TIME_PERIOD

Time period (as Date)

OBS_VALUE

Observation value

...

Additional dimension and metadata columns

Examples

## Not run: 
# Create and configure dataset
ds <- istat_dataset("139_176")
ds <- set_filters(ds,
  FREQ = "M",
  TIPO_DATO = c("ISAV", "ESAV"),
  PAESE_PARTNER = "WORLD"
)

# Get all data
data <- get_data(ds)

# Get data for a specific time range
data <- get_data(ds, start_period = "2020-01-01", end_period = "2023-12-31")

# Get only the last 12 observations
data <- get_data(ds, last_n_observations = 12)

## End(Not run)

Get description for a dimension from its codelist

Description

Get description for a dimension from its codelist

Usage

get_dimension_description(codelist_id)

Arguments

Value

Character string with the dimension description

Get available values for a dimension

Description

Retrieves all available values for a specific dimension of a dataset.

Usage

get_dimension_values(dataset, dimension_id)

Arguments

Value

A tibble with columns:

id

Value ID/code

name

Human-readable name (English)

Examples

## Not run: 
ds <- istat_dataset("139_176")
get_dimension_values(ds, "TIPO_DATO")

## End(Not run)

Get dimensions for a data structure

Description

Get dimensions for a data structure

Usage

get_dimensions(structure_id)

Arguments

Value

Named list of dimensions with their metadata

Parse language-specific name from XML

Description

Parse language-specific name from XML

Usage

get_name_by_lang(node, lang = "en")

Arguments

Value

Character string with the name in the specified language

Parse language-specific name from XML with namespace support

Description

Parse language-specific name from XML with namespace support

Usage

get_name_by_lang_ns(node, lang = "en", ns = NULL)

Arguments

Value

Character string with the name in the specified language

Create an ISTAT dataset object

Description

Creates a dataset object for a specific ISTAT dataflow. This object can be used to explore the dataset's structure, dimensions, and available values, and to set filters before retrieving data.

Usage

istat_dataset(dataflow_identifier)

Arguments

Value

A list with class "istat_dataset" containing:

df_id

Dataflow ID

version

Dataset version

df_description

Dataset description

df_structure_id

Data structure definition ID

dimensions

Named list of dimension information

filters

Named list of current filters (initialized to "." for all)

Examples

## Not run: 
# Create dataset by ID
ds <- istat_dataset("139_176")

# View dimensions
dimensions_info(ds)

# Get available values for a dimension
get_dimension_values(ds, "TIPO_DATO")

# Set filters
ds <- set_filters(ds, FREQ = "M", TIPO_DATO = c("ISAV", "ESAV"))

## End(Not run)

Quick data retrieval

Description

A convenience function that combines creating a dataset, setting filters, and retrieving data in one call.

Usage

istat_get(
  dataflow_id,
  ...,
  start_period = NULL,
  end_period = NULL,
  last_n_observations = NULL
)

Arguments

Value

A tibble containing the requested data

Examples

## Not run: 
# Quick retrieval with filters
data <- istat_get(
  "139_176",
  FREQ = "M",
  TIPO_DATO = "ISAV",
  PAESE_PARTNER = "WORLD",
  start_period = "2020-01-01"
)

## End(Not run)

Make a request to the ISTAT API

Description

Make a request to the ISTAT API

Usage

istat_request(path, accept = "application/xml", ...)

Arguments

Value

httr2 response object

Make a request and return CSV as tibble

Description

Make a request and return CSV as tibble

Usage

istat_request_csv(path, ...)

Arguments

Value

tibble with the data

Make a request and return XML with namespaces

Description

Make a request and return XML with namespaces

Usage

istat_request_xml(path, ...)

Arguments

Value

A list with 'xml' (the xml document) and 'ns' (the namespaces)

Get or set the API timeout

Description

The ISTAT API can be slow to respond, especially for large queries. This function allows you to get or set the timeout value in seconds. The default timeout is 300 seconds (5 minutes).

Usage

istat_timeout(seconds = NULL)

Arguments

Value

If seconds is NULL, returns the current timeout value. If seconds is provided, invisibly returns the previous timeout value.

Examples

# Get current timeout
istat_timeout()

# Set timeout to 10 minutes
istat_timeout(600)

# Set timeout back to default
istat_timeout(300)

Build SDMX URL key from filters

Description

Creates an SDMX-compliant filter key string from a list of dimension filters. Multiple values for a dimension are joined with "+", dimensions are separated by ".".

Usage

make_url_key(filters)

Arguments

Value

Character string with the SDMX filter key

Parse ISTAT time period strings to Date

Description

Handles various SDMX time period formats including:

• Annual: YYYY

• Monthly: YYYY-MM

• Quarterly: YYYY-Q1, YYYY-Q2, etc.

• Weekly: YYYY-W01, etc.

• Daily: YYYY-MM-DD

Usage

parse_time_period(x)

Arguments

Value

Date vector

Print method for istat_dataset

Description

Print method for istat_dataset

Usage

## S3 method for class 'istat_dataset'
print(x, ...)

Arguments

Value

Invisibly returns the input

Reset all filters to default (all values)

Description

Reset all filters to default (all values)

Usage

reset_filters(dataset)

Arguments

Value

The modified istat_dataset object with all filters reset to "."

Examples

## Not run: 
ds <- istat_dataset("139_176")
ds <- set_filters(ds, FREQ = "M")
ds <- reset_filters(ds)  # All filters back to "."

## End(Not run)

ISTAT Data Retrieval Functions

Description

Functions for retrieving data from ISTAT datasets

Search for datasets by keyword

Description

Searches available ISTAT datasets by keyword in their description. The search is case-insensitive.

Usage

search_dataset(keyword)

Arguments

Value

A tibble with matching datasets (same columns as all_available())

Examples

## Not run: 
# Search for datasets related to imports
import_datasets <- search_dataset("import")

# Search for population datasets
pop_datasets <- search_dataset("population")

## End(Not run)

Set filters for a dataset

Description

Sets dimension filters that will be used when retrieving data. Filter names should match dimension IDs (case-insensitive).

Usage

set_filters(dataset, ...)

Arguments

Value

The modified istat_dataset object

Examples

## Not run: 
ds <- istat_dataset("139_176")

# Set single values
ds <- set_filters(ds, FREQ = "M", PAESE_PARTNER = "WORLD")

# Set multiple values
ds <- set_filters(ds, TIPO_DATO = c("ISAV", "ESAV"))

## End(Not run)

ISTAT API Utility Functions

Description

XML parsing and helper functions

Extract attribute from XML node safely

Description

Extract attribute from XML node safely

Usage

xml_attr_safe(node, attr, default = NA_character_)

Arguments

Value

Character string

Extract text from XML node safely

Description

Extract text from XML node safely

Usage

xml_text_safe(node, xpath, default = NA_character_)

Arguments

Value

Character string