, , ,

SQL Database (RDBMS) Design for Data Engineers

Database design… hmmm. There is probably nothing more all over the board in tech. Data warehousing, analytics, OLTP… everyone with their own “defend this hill to the death” ideas. Kimball vs Inmon. Hmmm.. what to do, what to do? After defending my own hills to the death over the years and arguing over whiteboards I’ve come to a conclusion. The right answer is somewhere in the middle. Understanding a few basic design principals will help any data engineer master writing DDL for anything from a Data Warehouse to a high load OLTP systems… across all RDBMS platforms.

Read more
, ,

Httpx vs Requests in Python. Performance and other Musings.

Comparing the pypi requests vs httpx packages, who will fall on their face?

Someone recently brought up the new kid on the block, the httpx python package for http work of course. I mean the pypi package
requests has been the de-facto standard forever. Can it really be overthrown? Is this a classic case of “oh how the mighty have fallen”? I want to explore what the new httpx package has to offer, but mostly just …. which one is faster. That is what data engineers really care about.

Read more
, ,

Hey Google Cloud, ever heard of Boto3? Come on.

Why can’t GCP come up with their own Boto3?

First, let’s set the record straight. GCP is better than AWS. This will be clear to anyone who has used both services for a reasonable amount of time. GCP was built with the developer in mind, the services and tools offered work better, are cleaner, and way simplier. But, there is one thing that is totally annoying. Where is GCP’s answer to AWS’s Python Boto3 library? I mean seriously. Boto3 is the one stop shop to plugin and interact with pretty much every AWS service available, and the documentation is reasonable. Seriously GCP, where you at?

Read more
, ,

How Chuck Norris Proved Async in Python isn’t Worthy.

Chuck Norris proved that Async Python is slower and suckier than Process and Thread Pools.

There are some things I will never understand. Async in Python is one of them. Yes, sometimes I use it, but mostly because I’m bored and we all should have some kind of penance. Async in mine. It’s slow, confusing, other people get mad at you when they have to debug your Async code. I’ve always wondered why anyone would choose a single Python process to do a bunch of work instead of …… more then one? I decided there is only one person who could solve the interwebs Async arguments, Chuck Norris.

Read more