Skip to contents

Mostly used internally by cas_download.

Usage

cas_download_httr(
  download_df = NULL,
  index = FALSE,
  index_group = NULL,
  overwrite_file = FALSE,
  wait = 1,
  create_folder_if_missing = NULL,
  pause_base = 2,
  pause_cap = 256,
  pause_min = 4,
  terminate_on = 404,
  retry_times = 16,
  db_connection = NULL,
  disconnect_db = FALSE,
  sample = FALSE,
  file_format = "html",
  user_agent = NULL,
  download_again_if_status_is_not = NULL,
  ...
)

Arguments

download_df

A data frame with four columns: id, url, path, type.

index

Logical, defaults to FALSE. If TRUE, downloaded files will be considered index files. If not, they will be considered contents files. See Readme for a more extensive explanation.

overwrite_file

Logical, defaults to FALSE.

wait

Defaults to 1. Number of seconds to wait between downloading one page and the next. Can be increased to reduce server load, or can be set to 0 when this is not an issue.

retry_times

Defaults to 10. Number of times to retry download in case of errors.

db_connection

Defaults to NULL. If NULL, uses local SQLite database. If given, must be a connection object or a list with relevant connection settings (see example).

disconnect_db

Defaults to TRUE. If FALSE, leaves the connection to database open.

sample

Defaults to FALSE. If TRUE, the download order is randomised. If a numeric is given, the download order is randomised and at most the given number of items is downloaded.

user_agent

Defaults to NULL. If given, passed to download method.

...

Passed to cas_get_db_file().

Value

Invisibly returns the full httr response.