Tech Tips

Thursday, August 19, 2021

PC Build

Summary

I'm going to present a series of pics depicting a new PC build I completed this week. This replaces the PC I built ~8 years ago (and is still fully operational). That PC had 4 CPU cores and 16 GB RAM. This one has 16 cores and 64 GB, so a 4x upgrade.

Equipment List

Pic below with the boxed components used in this build.

Motherboard - Gigabyte Aorus Elite. I've had good luck with Gigabyte MBs in the past, so I'm a repeat customer.
CPU - AMD Ryzen 5950X. 16 CPU cores, 32 logical cores with Simultaneous Multithreading (SMT).
Case - Corsair Obsidian 750D. Again, another case of repeat success + customer. This one has thorough ventilation + 3x140mm fans - 2 in front, 1 behind.
Drive - Samsung 980 SSD 1 TB. I consider Samsung the top end on drives. This one can be had at a good price.
Power Supply - Seasonic Focus PX-750. More repeat success + customer. The Seasonic PSU I bought 8 years ago is still running.
CPU Cooler - Noctua NH-D15 Chromax.Black. This is an air cooler - and it's a beast. I really like how this company makes fans. Super quiet.
GPU - Biostar Radeon RX 550. Contrary to the rest of this build - this component is on the low end. It's old GPU tech. Unfortunately, the hoards of Bitcoin miners out there have pushed the prices of current generation GPUs beyond what I'm willing to pay. I'm not a gamer, so I don't really need the high-end anyway.
RAM - G.Skill RipJaws V Series 64GB. More repeat success + customer. The Aorus board will take 4 x 32GB (128 GB), so I'm filling it to half at this point.

Unboxed pic of the same components below.

BIOS Updating

The BIOS on these boards needs to be updated before you even install the CPU. Gigabyte makes that easy with a feature called Q-Flash plus. Steps:

Download the current BIOS rev from Gigabyte's site
Rename it to 'GIGABYTE.bin' and copy it to a USB drive.
Put that drive in the MB USB slot that is tagged for this.
Power up the board.
Push the Q-flash button on the board. The associated MB LED will light up until the update is complete.

Pic below of me flashing this board.

Seating the CPU

Pic below of the Ryzen chip seated on the AM4 socket of this board.

Installing the CPU Cooler

The stock cooler retainer brackets on this board need to be removed and replaced with the Noctua bracket. The Noctua cooler includes thermal paste, so no need to buy any separately. Pic below of the brackets and paste applied.

Pic below of the Noctua cooler now installed.

Note about Fan Headers

This board has a total of 4 Fan Headers. For this build, there are 3 fans in the case and 2 on the CPU cooler. That equals 5. That would be a problem except for the fact that the Noctua kit includes a Y-cable for splicing two fans to one header. I'm showing a pic below of that cable on the 2 CPU fan cables. I didn't actually install it that way due to physical distances between fans and headers. Instead, I used that Y-cable to combine the case's front two fans into 1 and the connected them to one of the SYS FAN headers. I connected each of the CPU cooler fans to the two CPU fan headers. The rear case fan I connected to the rear SYS FAN header.

Note that I've also installed both CPU fans on the cooler in this pic. This works for this build.

RAM Installation

Pic below of both RAM sticks in place. They fit under the CPU cooler + fans. For only 2 sticks, you use slots A2 and B2 on this board.

NVM Drive Installation

Pic below of NVM drive in place on this board. There's a piece of plastic wrap on the M2 heat dissipater that you need to remove before final seating.

GPU Installation

My low-end board on the main PCI slot of the MB below.

Motherboard Seating into Case

Pics below of the full package installed in the case.

Complete System

Bios screenshot of completed system below:

Ubuntu System Monitor screen-shot

Completed build in its native habitat.

Wednesday, June 30, 2021

Google Cloud Devops

Summary

This post will demonstrate the usage of Google Cloud's serverless deployment pipeline - Cloud Build. The use case for this will be a fairly simple Python app that exposes a REST interface via Flask + NLTK for tokenization of text.

Overall Architecture

The diagram below depicts the Cloud Build pipeline.

Python Application Organization

Cloud Build Steps

Cloud Build is orchestrated from a cloudbuild.yaml file. Example code below with associated diagram.

steps:
  #Unit Test
  - name: python
    entrypoint: /bin/sh
    args: ["-c", 
     "pip install -r requirements.txt &&\ 
     python -c \"import nltk; nltk.download('popular', download_dir='/home/nltk_data')\" &&\
     export NLTK_DATA=/home/nltk_data &&\ 
     python -m unittest"] 
  
  #Docker Build
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 
            'us-central1-docker.pkg.dev/$PROJECT_ID/$_REPO_NAME/cleaner', '.']
  
  #Docker push to Google Artifact Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push',  'us-central1-docker.pkg.dev/$PROJECT_ID/$_REPO_NAME/cleaner']
  
  #Deploy to Cloud Run
  - name: google/cloud-sdk
    args: ['gcloud', 'run', 'deploy', 'cleaner', 
           '--image=us-central1-docker.pkg.dev/$PROJECT_ID/$_REPO_NAME/cleaner', 
           '--region', 'us-central1', '--platform', 'managed', 
           '--allow-unauthenticated']

Screenshots of Results

Cloud Build

Artifact Registry

Cloud Run

Source

https://github.com/joeywhelan/devops-demo

Saturday, April 17, 2021

Python Virtual Environments + VSCode

Summary

This is a brief post on setting up a Python3 virtual environment. A virtual environment enables you to maintain a clean set of Python modules for your specific project.

Step 1

Create a folder for your project. For this demo, I'm calling it 'envdemo'.

Step 2

Open that folder in VSCode and start a terminal session.

Step 3

Execute the python3 command to create a virtual environment in that folder.

Step 4

Go to View, Command Palette, Select Python Interpreter and then select the Python interpreter from your new virtual environment.

Step 5

Open a new terminal session. A Workspace (.vscode) has been created for this project and the default interpreter is set to the one in your new virtual environment. Also, note that no Python modules have been installed yet in this environment.

Step 6

Proceed with installing the Python modules necessary for this particular project and develop code.

Sunday, February 28, 2021

RFC 7523 Demo

Summary

I'll be covering the usage of the RFC 7523 authorization grant type in this post. I'll create a server-side implementation as a Google Cloud Function and a simple client in Node.js.

Architecture

The implementation consists of a Google Cloud Function that is triggered on an HTTP POST request. The request has a form-urlencoded body consisting of the grant_type parameter and an assertion with the JSON web token (JWT) - per the RFC. The response is a JSON object consisting of the token type, bearer token, and the unencoded version of the original JWT received. That last item isn't required, but I included it to aid in troubleshooting.

Server-side Snippet

As discussed, the server is implemented as a GCF. For a Node implementation, that consists of an Express server. The express-jwt middleware is used to implement the JWT handling.

app.use(expressJwt({
    secret: sharedKey,
    issuer: issuer,
    audience: audience,
    algorithms: ['HS256', 'HS384', 'HS512'],    
    requestProperty: 'token',
    getToken: (req) => {
        return req.body.assertion;
    }
}));

app.post('/rfc7523', (req, res) => {
    if (req.token) {
        console.log(`Received token: ${JSON.stringify(req.token)}`);
        const alg = 'HS512'
        const payload = { 
            "iss": 'oauth issuer',
            "sub": 'oauth authority',
            "aud": 'm2mclient',
            "exp": Math.round(Date.now()/1000 + 3) //3 second expiration
        };
        const accessToken = jwt.sign(payload, privateKey, {algorithm: alg});
            
        res.status(200)
        .json({
            token_type: 'bearer',
            rec_token: req.token,
            access_token: accessToken
        });
    }
    else {
        res.status(400).json({error: 'no token found'});
    }
});

Client-side Snippet

(async () => {
    const payload = { 
        "iss": issuer,
        "sub": subject,
        "aud": audience,
        "exp": Math.round(Date.now()/1000 + 3) //3 second expiration        
    };
    const alg = 'HS512'
    const token = jwt.sign(payload, sharedKey, {algorithm: alg});
    const authGrant = encodeURI('urn:ietf:params:oauth:grant-type:jwt-bearer');
    const response = await fetch(url, {
        method: 'POST',
        headers: {'Content-Type': 'application/x-www-form-urlencoded'},
        body: `grant_type=${authGrant}&assertion=${token}`
    });
    
    const json = await response.json();
    console.log(`Results: ${JSON.stringify(json, null, 4)}`);
})();

Results

Results: {
...
    "access_token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJvYXV0aCBpc3 - abbreviated"
}

Source

https://github.com/joeywhelan/rfc7523demo

Sunday, February 21, 2021

Simulation with Simpy

Summary

In this post, I'll be using the Simpy Python framework to create a simulation model. I'll cover the base case here of generating requests to a finite set of resources. I'll compare the simulation results to the expected/theoretical results for an Erlang B model.

Architecture

The diagram below depicts the base simulation model. A Request Generator sends a stream of requests at an interval corresponding to a Poisson process. An intermediary process, the Regulator, makes a decision based on resource availability of where to route the request. If all resources are busy, the request is blocked. Otherwise, the resource assumes a position in the queue and is subsequently serviced by a worker for a period of time representing a Normal distribution on the average handle time.

Code Snippet

def scenario_one():
    rand: Random = Random()
    rand.seed(RANDOM_SEED)
    env: Environment = Environment()
    distributor: Distributor = Distributor(rand, env, POSITIONS, 0, HANDLE_TIME_MEAN, 0, QUEUE_COST, HANDLE_COST)
    regulator: Regulator = Regulator(env, distributor)
    requestGenerator: RequestGenerator = RequestGenerator(rand, env, regulator, 0, 0, STEADY_RATE)
    env.run(until=TOTAL_DURATION)
    print(f'***  Scenario 1: ErlangB Sanity Check.  Partial simulation: no workers, no surge, no deflection ***')
    print(f'Total Requests: {requestGenerator.total_requests}')
    print(f'Total Queued Requests: {distributor.total_queued}')
    print(f'Total Serviced Requests: {distributor.total_serviced}')
    print(f'Total Deflected Requests: {regulator.total_deflected}')
    print(f'Total Blocked Requests: {regulator.total_blocked}')
    print(f'Max Queue Depth: {distributor.max_positions_depth}')
    print(f'Max Resources Consumed: {distributor.max_resources}')
    print(f'Probability of Blockage: {regulator.total_blocked/requestGenerator.total_requests}')
    print(f'')

Results

Below are the results with 20 requests/sec rate and 15 minute average handle time. The Erlang B blocking calculation for those parameters = 0.666694

***  Scenario 1: ErlangB Sanity Check.  Partial simulation: no workers, no surge, no deflection ***
Total Requests: 864377
Total Queued Requests: 288000
Total Serviced Requests: 0
Total Deflected Requests: 0
Total Blocked Requests: 576376
Max Queue Depth: 6000
Max Resources Consumed: 6000
Probability of Blockage: 0.666810893857657

Sunday, January 10, 2021

Google Cloud Healthcare - Analytics

Summary

This post is a continuation of my previous on the Google Healthcare API. In this post, I'll push the FHIR datastore into Google's data warehouse - BigQuery. Once in BigQuery, the data can then be subjected to traditional analytics tools (SQL queries) and visualized with Google's report/dashboard tool - Data Studio. For the purposes of these demos, I extended the Synthea-generated recordsets to 50 patient bundles.

Architecture

Below is a diagram of the cloud architecture. FHIR data (JSON-based) is transformed into relational database tables on BigQuery. SQL queries can then be created to analyze the data. Finally, the output of those queries can be saved as Views and then presented in charts in Data Studio.

BigQuery Execution

FHIR Export

Below is the gcloud command-line to export an FHIR datastore to BigQuery. This is a one-time export; however, it is possible to configure a continuous stream of updates from the FHIR store to BigQuery as well.

gcloud healthcare fhir-stores export bq $FHIR_STORE_ID \
  --dataset=$DATASET_ID \
  --location=$LOCATION \
  --bq-dataset=bq://$PROJECT_ID.$BIGQUERY_DATASET_ID \
  --schema-type=analytics

Query 1 - Top Ten Medications

At this point, a relational database is created within BigQuery and ready for analytics. Below are a query and its output to find the top 10 prescribed meds within the FHIR datastore.

Query 2 - Demographics

Below is a query that provides a bucketing of the patient age groups.

Query 3 - Top Ten Conditions

Below is a query to derive the top 10 conditions within the patient population.

Views

I then created views for each of these queries. Those views will be used for the presentation layer of the output in Data Studio. Below is the view of the demographics query.

Data Studio Configuration

Now that the views are set up in BigQuery, it's now possible to create visualizations of them using Data Studio. Below are the steps to do that.

Create a blank report

Select BigQuery as the data source

Select the BigQuery View

Configure the presentation

Choose the chart type

Output

Top Ten Medications

Demographics - Age Distribution

Top Ten Conditions

Source

https://github.com/joeywhelan/clinicsimulation/tree/main/analytics

Friday, January 1, 2021

Google Cloud Healthcare - Clinic Simulation

Summary

In this post, I'll demonstrate the usage of the Google Cloud Healthcare API via a contrived scenario. The simulation here is a clinic in Colorado that is taking on new patients. Each patient brings with them a health history that is contained in electronic health records in an FHIR format. These FHIR records are pushed into the Healthcare API and then available for search, analytics, etc. I'll be using Synthea to generate artificial records.

Scenario

Below is a depiction of the simulation at a logical level.

Architecture

Below is the overall Google Cloud architecture. FHIR records are generated from Synthea and then uploaded to Cloud Storage. Google Cloud Scheduler is used to simulate a stream of patients arriving at the clinic. Cloud Scheduler triggers a Cloud Function via Pub/Sub. That function pulls down a random number of FHIR patient transaction bundles and then executes them to load them into Cloud Healthcare.

Patient Record Generation

As mentioned previously, I'm using Synthea to generate FHIR records. There are three (3) releases of the FHIR standard 'supported' by the Healthcare API. Based on my experience here, Google is not supporting the latest release (R4) adequately. There were enough issues with Synthea-generated R4 bundles with Google that I gave up on troubleshooting them and reverted to the previous FHIR rev - STU3. Those work.

Below is the config I used for this simulation. This yields STU3 FHIR resource bundles with all living patients. The default Synthea config will generate a random number of deceased patients. In the spirit of this simulation, it's not likely any deceased patients will be admitting themselves to the clinic.

generate.only_alive_patients = true

exporter.fhir.transaction_bundle = true
exporter.fhir.export = false
exporter.fhir_stu3.export = true
exporter.fhir_dstu2.export = false

exporter.hospital.fhir.export = false
exporter.hospital.fhir_stu3.export = true
exporter.hospital.fhir_dstu2.export = false

exporter.practitioner.fhir.export = false
exporter.practitioner.fhir_stu3.export = true
exporter.practitioner.fhir_dstu2.export = false

Below is the command line I used (Developer) and a snippet of the output. I generated ten living patients in Colorado.

./run_synthea -p 10 Colorado -c ./config.txt

Population: 10
Seed: 1609530634098
Provider Seed:1609530634098
Reference Time: 1609530634098
Location: Colorado
Min Age: 0
Max Age: 140
5 -- Bradford382 Marks830 (1 y/o M) Security-Widefield, Colorado 
4 -- Laveta191 Becker968 (2 y/o F) Denver, Colorado 
2 -- Valene773 Shanahan202 (17 y/o F) Denver, Colorado 
6 -- Valrie435 Cruickshank494 (47 y/o F) Highlands Ranch, Colorado 
1 -- Chance908 Murray856 (56 y/o M) Pueblo, Colorado 
3 -- Lashawnda573 Hettinger594 (70 y/o F) Thornton, Colorado 
8 -- Shin962 Bahringer146 (69 y/o F) Denver, Colorado 
7 -- Johnie961 Padberg411 (68 y/o M) Thornton, Colorado 
10 -- Bridget51 Leffler128 (24 y/o F) Louisville, Colorado 
9 -- Carlos172 González124 (55 y/o M) Denver, Colorado 
Records: total=10, alive=10, dead=0

Google Cloud Configuration

Screenshots below of the various upfront configuration activities that were necessary for this simulation:

Cloud Storage

Cloud Healthcare

Cloud Pub/Sub

Cloud Scheduler

Cloud Function

Hospital and Practitioner Upload

Synthea generates three types of FHIR transaction bundles for a given synthetic load: Patient, Hospital, and Practitioner. Patient resources reference Hospital and Practitioner resources. Practitioner resources reference Hospital resources. Below are the CURL commands I used to upload the Hospital and Practitioner bundles. Note that I'm using 'v1beta1' of the Healthcare API. V1 doesn't handle conditional references properly. The Synthea bundles have conditional references in them.

curl -X POST \
    -H "Content-Type: application/fhir+json; charset=utf-8" \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    --data @hospitalInformation1609530634625.json \
    "https://healthcare.googleapis.com/v1beta1/projects/clinic-simulation/locations/us-central1/datasets/clinicset/fhirStores/clinicSTU3/fhir"


curl -X POST \
    -H "Content-Type: application/fhir+json; charset=utf-8" \
    -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
    --data @practitionerInformation1609530634625.json \
    "https://healthcare.googleapis.com/v1beta1/projects/clinic-simulation/locations/us-central1/datasets/clinicset/fhirStores/clinicSTU3/fhir"

Code Snippets

Cloud Function for Executing Transaction Bundles

/**
 * Reads a FHIR bundle from GCS and then loads it to a Google Healthcare FHIR store
 * @param {File} file - GCS File object
 * @return {Promise} 
 * @throws {Error} propagates exceptions
 */
async function loadBundle(file) {   
    const auth = await google.auth.getClient({
        scopes: ['https://www.googleapis.com/auth/cloud-platform'],
    });
    google.options({
        auth, 
        headers: {
            'Content-Type': 'application/fhir+json'
        }
    });

    let bundle = '';
    return new Promise((resolve, reject) => {
        file.createReadStream()
        .on('data', (chunk) => {
            bundle += chunk;
        })
        .on('end', async () => {
            const request = { 
                parent: BASE_URL,
                type: 'Bundle',
                requestBody: JSON.parse(bundle),
            };
            let response;

            try {
                response = await healthcare.projects.locations.datasets.fhirStores.fhir.executeBundle(request);
                resolve(response.statusText);
            }
            catch(err) {
                reject(err);
            }
        })
        .on('error', (err) => {
            reject(err);
        });
    });
}

Search Function

Code below searches the FHIR store for male patients over age 60.

async function search() { 
    const auth = await google.auth.getClient({
        scopes: ['https://www.googleapis.com/auth/cloud-platform'],
    });
    google.options({
        auth, 
        params: {
            gender:'male',
            birthdate: 'lt1961-01-01'
        }
    });
  
    const request = { 
        parent: BASE_URL, 
        resourceType : 'Patient'
    };
    const response = await healthcare.projects.locations.datasets.fhirStores.fhir.search(request);
    
    return response.data.entry;
}

Execution

Log snippet below for the Generator function executing patient transaction bundles from Cloud Storage.

The abbreviated output below from the search function mentioned previously. Recall there was only 1 male patient over 60 in the ten patients that were loaded.

     "name": [
        {
          "family": "Padberg411",
          "given": [
            "Johnie961"
          ],
          "prefix": [
            "Mr."
          ],
          "use": "official"
        }
      ],
      "resourceType": "Patient",


Result count: 1

Source

https://github.com/joeywhelan/clinicsimulation