You can install CDECRetrieve the usual way,
# for stable version
install.packages("CDECRetrieve")
# for development version
devtools::install_github("flowwest/CDECRetrieve")
The goal for CDECRetrieve is to create a workflow for R users using CDEC data, we believe that a well defined workflow is easier to automate and less prone to error (or easier to catch errors). In order to do this we create “services” out of different endpoints available through the CDEC site. A lot ideas in developing the package came from using dataRetrieval
from USGS and the NOAA CDO api.
We start by first exploring locations of interest. The CDEC site provides a web form with a lot of options,
cdec station search
The pakcage exposes this functionallity through cdec_stations()
. Although it doesn’t (currently) map all options in the web form it does so for the most used, namely, station id, nearby city, river basin, hydro area and county. At least one of the parameters must be supplied, and combination of these can be supplied to refine the search.
library(CDECRetrieve)
cdec_stations(station_id = "kwk") # return metadata for KWK
#> # A tibble: 0 x 9
#> # ... with 9 variables: station_id <chr>, name <chr>, river_basin <chr>,
#> # county <chr>, longitude <lgl>, latitude <lgl>, elevation <lgl>,
#> # operator <lgl>, state <chr>
# show all locations near san francisco, this returns a set of
# CDEC station that are near San Francisco
cdec_stations(nearby_city = "san francisco")
#> # A tibble: 3 x 9
#> station_id name river_basin county longitude latitude elevation operator
#> <chr> <chr> <chr> <chr> <dbl> <dbl> <int> <chr>
#> 1 cx2 dail… sf bay san f… -122. 37.8 0 CA Dept…
#> 2 ggt gold… sf bay san f… -122. 37.8 0 Nationa…
#> 3 sfn san … sf bay san f… -122. 37.8 150 Nationa…
#> # ... with 1 more variable: state <chr>
# show all location in the sf bay river basin
cdec_stations(river_basin = "sf bay")
#> # A tibble: 24 x 9
#> station_id name river_basin county longitude latitude elevation
#> <chr> <chr> <chr> <chr> <dbl> <dbl> <chr>
#> 1 lfy lafa… sf bay contr… -122. 37.9 465
#> 2 mas main… sf bay santa… -1000. 100.0 9,999
#> 3 spb san … sf bay contr… -122. 37.9 330
#> 4 ttk term… sf bay santa… -122. 37.4 650
#> 5 mrh mars… sf bay contr… -122. 37.9 740
#> 6 cx2 dail… sf bay san f… -122. 37.8 0
#> 7 acm arro… sf bay marin -123. 37.9 3
#> 8 paa palo… sf bay santa… -122. 37.4 7
#> 9 sww swee… san antoni… santa… -121. 37.4 2,150
#> 10 ggt gold… sf bay san f… -122. 37.8 0
#> # ... with 14 more rows, and 2 more variables: operator <chr>, state <chr>
# show all station in Tehama county
cdec_stations(county = "tehama")
#> # A tibble: 45 x 9
#> station_id name river_basin county longitude latitude elevation
#> <chr> <chr> <chr> <chr> <dbl> <dbl> <chr>
#> 1 blb blac… stony cr tehama -122. 39.8 426
#> 2 rbf red … sacto vly … tehama -122. 40.2 353
#> 3 sh1 shee… sacramento… tehama -123. 39.5 6,500
#> 4 cot cott… cottonwood… tehama -122. 40.4 364
#> 5 ctn cott… cottonwood… tehama -123. 40.3 3,400
#> 6 mlm mill… sacramento… tehama -122. 40.1 385
#> 7 mnr mine… sacto vly … tehama -122. 40.4 4,875
#> 8 sbb sacr… sacto vly … tehama -122. 40.3 186
#> 9 vin sacr… sacramento… tehama -122. 39.9 185
#> 10 crg corn… sacramento… tehama -122. 39.9 294
#> # ... with 35 more rows, and 2 more variables: operator <chr>, state <chr>
Since we are simply exploring for locations of interest, it may be useful to map these for visual inspection. CDECRetrieve provides a simple function to do exactly this map_stations()
.
library(magrittr)
library(leaflet)
cdec_stations(county = "tehama") %>%
map_stations()
The same can be done with leaflet functions
d <- cdec_stations(county = "tehama")
leaflet(d) %>%
addTiles() %>%
addCircleMarkers(label=~station_id) #psk is way off here
After exploring stations in a desired location. We can start focusing on the datasets available at the locations.
station <- "sha"
cdec_datasets("sha")
#> # A tibble: 21 x 6
#> sensor_number sensor_name sensor_units duration start end
#> <int> <chr> <chr> <chr> <date> <date>
#> 1 2 precipitatio… inches daily 2003-10-01 2018-09-10
#> 2 2 precipitatio… inches monthly 1953-10-01 2018-09-10
#> 3 6 reservoir el… feet daily 1985-01-01 2018-09-10
#> 4 6 reservoir el… feet hourly 1993-12-09 2018-09-10
#> 5 8 full natural… cfs daily 1987-05-31 2018-09-10
#> 6 15 reservoir st… af daily 1985-01-01 2018-09-10
#> 7 15 reservoir st… af hourly 1994-06-24 2018-09-10
#> 8 15 reservoir st… af monthly 1953-10-01 2018-09-10
#> 9 22 reservoir st… af daily 1993-10-03 2018-09-10
#> 10 23 reservoir ou… cfs daily 1987-01-05 2018-09-10
#> # ... with 11 more rows
Since all of these functions return a tidy dataframe we can make use of the dplyr
to filter, mutate and explore. Here we look for datasets in Shasta that report a storage
library(magrittr)
cdec_datasets("sha") %>%
dplyr::filter(grepl("storage", sensor_name))
#> # A tibble: 5 x 6
#> sensor_number sensor_name sensor_units duration start end
#> <int> <chr> <chr> <chr> <date> <date>
#> 1 15 reservoir sto… af daily 1985-01-01 2018-09-10
#> 2 15 reservoir sto… af hourly 1994-06-24 2018-09-10
#> 3 15 reservoir sto… af monthly 1953-10-01 2018-09-10
#> 4 22 reservoir sto… af daily 1993-10-03 2018-09-10
#> 5 94 reservoir top… af daily 2000-10-24 2018-09-10
Take note of the sensor number, and duration, these will be needed for querying data in the next section.
Now that we have a location, parameter of interest and duration we can start to query for actual data.
sha_storage_daily <- cdec_query(station = "sha", sensor_num = "15",
dur_code = "d", start_date = "2018-01-01",
end_date = Sys.Date())
sha_storage_daily
#> # A tibble: 253 x 5
#> agency_cd location_id datetime parameter_cd parameter_value
#> <chr> <chr> <dttm> <chr> <dbl>
#> 1 CDEC SHA 2018-01-01 00:00:00 15 3203249
#> 2 CDEC SHA 2018-01-02 00:00:00 15 3202064
#> 3 CDEC SHA 2018-01-03 00:00:00 15 3203723
#> 4 CDEC SHA 2018-01-04 00:00:00 15 3206566
#> 5 CDEC SHA 2018-01-05 00:00:00 15 3210358
#> 6 CDEC SHA 2018-01-06 00:00:00 15 3215097
#> 7 CDEC SHA 2018-01-07 00:00:00 15 3217003
#> 8 CDEC SHA 2018-01-08 00:00:00 15 3229391
#> 9 CDEC SHA 2018-01-09 00:00:00 15 3237014
#> 10 CDEC SHA 2018-01-10 00:00:00 15 3242032
#> # ... with 243 more rows
Once again the the data is in a tidy form.
We can plot with ggplot2
library(ggplot2)
sha_storage_daily %>%
ggplot(aes(datetime, parameter_value)) + geom_line()