-
Notifications
You must be signed in to change notification settings - Fork 0
Installation
After cloning this repository, a few things must be done.
git clone https://github.com/rkiddy/ca_hhs.git
cd ca_hhs
First, set up the python environments. I have two, one in .venv and one in .senv. The first supports most of the fetching and saving activity. The second supports the selenium functionality that is necessary to get some of the data.
Most of the pages at CalHHS have a 'Download All' button and, for those, it is fairly easy to use the requests module to get this files. Some of the datasets have the 'Download All' button disabled. Some of the datasets have the 'Download All' button enabled and it is broken. For these two, one must use selenium. There are links on the dataset page, but they do not link to the resources. They link to pages, each of which contains a link to one resource.
I use mysql (right now, Ver 8.0.42) and set up two databases. The ca_hhs database contains the data from the datasets and a ca_hhs_meta database contains tables which track the datasets. The following assumes that you have a .my.cnf file set up in your home directory which allows you to type "mysql" and get into your databases.
These databases can be created thusly:
wget https://opencalaccess.com/hcai/ca_hhs_meta.sql
echo "create database ca_hhs;" | mysql
echo "create database ca_hhs_meta;" | mysql
mysql < ca_hhs_meta.sql