Working with Split Part in Postgres

Intro Postgresql split_part function allows us to split a string in to a number of substrings. This function help when formatting columns for a specific query. In this article, we will learn how to…

Working with Schemas in Postgres

Intro Postgresql provides Schemas to organize databases. This allows you to create separation amongst your app and limit access to parts of your instance. In this article, we will learn how to use…

Working with Identity Columns in Postgres

Intro Postgresql provides the GENERATED AS IDENTITY constraint to allow users a way to create unique keys on data. This is similar to the SERIAL data type, but has a few extra options. In this article…

Working with Sequences in Postgres

Intro Sequences allow you to define number generators in Postgresql. They are often use to defined primary keys on table, but are heavily customizable. In this article, we will learn how to use…

Working with Truncate Table in Postgres

Intro The Truncate Table allows us to delete large tables quickly. If you want to delete all records, the DELETE command will be slower than TRUNCATE. This is because TRUNCATE will skip many checks…

How to Rename a Column in Postgres

Intro Postgresql provides the RENAME COLUMN clause when you nee to change the name of a column in an already created table. In this article, we will see an example how how to use the Rename Column in…

Working with Change Column Type in Postgres

Intro Often when developing we will want to change the column types on an existing table. We can do this using the ALTER TABLE and TYPE clauses. In this article, we will learn how to change column…

How to Install SQL Server with Docker

Intro SQL Server is one of the most widely used databases. Microsoft supports this database development and SQL Server has grown to support much more than just SQL. It also can scale up to many…

Working with Drop Column in Postgres

Intro The Add Drop statement allows us to remove columns from an existing table in SQL. This is a common statement to use as you are developing your app. In this article, we will learn how to use Drop…

Working with Add Column in Postgres

Intro The Add Column statement allows us to add columns to an existing table in SQL. This is a common statement to use as you are developing your app. In this article, we will learn how to use Add…

Working with Add Column in MySQL

Intro The Add Column statement allows us to add columns to an existing table in SQL. This is a common statement to use as you are developing your app. In this article, we will learn how to use Add…

Working with Alter Table in Postgres

Intro The Alter Table allows us to make changes to our SQL tables after they were designed. This is common in applications that are continuously evolving and being built on. With alter table we can…

Working with Alter Table in MySQL

Intro The Alter Table allows us to make changes to our SQL tables after they were designed. This is common in applications that are continuously evolving and being built on. With alter table we can…

How to Create Services in Kubernetes

Intro In Kubernetes, a service is a set of pods that provides and IP and/or DNS name for the pods to be accessed. This is the entry point for you app. Say you launch a pod or pods with a python api…

How to Use Namespaces in Kubernetes

Intro Namespaces allow you to section off parts of your Kubernetes cluster and resources. This prevents name collisions and incorrect usage. For example, you may have many pods named and deploy a…

Working with Create Temporary Table in Postgres

Intro The CREATE TEMPORARY TABLE statement allows us to create a short-lived table then is removed at the end of our session. This is helpful when doing some local calculations that you don't want to…

Working with Create Temporary Table in MySQL

Intro The CREATE TEMPORARY TABLE statement allows us to create a short-lived table then is removed at the end of our session. This is helpful when doing some local calculations that you don't want to…

Working with Create Table in Postgres

Intro The CREATE TABLE statement is a fundamental statement in SQL that allows you to create databases in SQL. When using this statement, you will be able to detail columns, constraints and other…

Working with Create Table in MySQL

Intro The CREATE TABLE statement is a fundamental statement in SQL that allows you to create databases in SQL. When using this statement, you will be able to detail columns, constraints and other…

Working with Update Join in Postgres

Intro The UPDATE statement with the Join allows us to change rows in a table based on joined data. Often you will want to connect two tables and updated based on conditions form the joined result. In…

Working with Update Join in MySQL

Intro The UPDATE statement with the Join allows us to change rows in a table based on joined data. Often you will want to connect two tables and updated based on conditions form the joined result. In…

Working with Replace in MySQL

Intro The statement allows us to overwrite a new row when we insert. When using REPLACE, MySQL will delete a row if there is a duplicate, then insert the new row. In this article, we will learn how…

How to Create Jobs in Kubernetes

Intro Jobs in Kubernetes allow us to run one offs or recurring tasks using containers in our cluster. Common use cases for this are running one off reports from a database query, running tests during…

Working with Delete Join in Postgres

Intro The allows us to delete data from multiple tables using a join. Often we will want o connect multiple tables, then delete rows. In this article, we will learn how to use in PostgreSQL. The…

Working with Delete Join in MySQL

Intro The allows us to delete data from multiple tables using a join. Often we will want o connect multiple tables, then delete rows. In this article, we will learn how to use in MySQL. The Syntax…

Working with Delete Cascade in Postgres

Intro The action allows us to set up an action on a relationship that will delete related rows when the parent is deleted. For example, if we have employees and salaries, we can set up our database…

Working with Delete Cascade in MySQL

Intro The action allows us to set up an action on a relationship that will delete related rows when the parent is deleted. For example, if we have employees and salaries, we can set up our database…

How to Create Pods in Kubernetes

Intro Creating pods is an every day task when working wit Kubernetes. Kubernetes provides a simple way to defined pods using yaml and to deploy them with the kubectl command. In this article, we will…

How to import a csv into Postgres table

Intro Often we are given a csv of data, and we would like to now how to copy that data directly to a SQL table. Using the COPY command, we can do that easily. In this article, we will learn how to use…

Working with Drop Database in Postgres

Intro The DROP DATABASE statement allows you to completely delete a database an all its contents. In this article, we will learn how to drop database in PostgreSQL. The Syntax The basic syntax of DROP…

Working with Drop Database in MySQL

Intro The DROP DATABASE statement allows you to completely delete a database an all its contents. In this article, we will learn how to drop database in MySQL. The Syntax The basic syntax of DROP…

How to find php.ini on Windows

Intro The BETWEEN operator allows you to filter for values in between two other values. For example, we could select items that are in a specific date range or houses in between a price range. In this…

Working with Update in Postgres

Intro The UPDATE statement allows us to change rows in a table. This is one of the main statements you will use while working with SQL. In this article, we will learn how to use UPDATE in MySql. The…

Working with Update in MySQL

Intro The UPDATE statement allows us to change rows in a table. This is one of the main statements you will use while working with SQL. In this article, we will learn how to use UPDATE in MySql. The…

Working with Insert Into Ignore in Postgres

Intro When inserting multiple rows, but default SQL will stop and return an error if there is an issue and no rows are inserted. If we use the INSERT IGNORE statement we can skip errors and insert all…

Working with Insert Ignore in MySQL

Intro When inserting multiple rows, but default SQL will stop and return an error if there is an issue and no rows are inserted. If we use the INSERT IGNORE statement we can skip errors and insert all…

Working with Insert Into Select in Postgres

Intro When using the INSERT statement, we sometimes want to insert data from one table into another. We can use a SELECT clause after our INSERT to achieve this. In this article, we will learn how to…

Working with Insert Into Select in MySQL

Intro When using the INSERT statement, we sometimes want to insert data from one table into another. We can use a SELECT clause after our INSERT to achieve this. In this article, we will learn how to…

Working with Insert in Postgres

Intro The INSERT statement allows us to insert data into our tables. It is one of the main operations you will use. In this article, we will learn how to use Insert in MySQL. The Syntax The basic…

Working with Insert in MySQL

Intro The INSERT statement allows us to insert data into our tables. It is one of the main operations you will use. In this article, we will learn how to use Insert in MySQL. The Syntax The basic…

Working with Intersection in Postgres

Intro The INTERSECT operator allows us to join two queries together and only select the rows they share in common. This is similar to UNION except we only get rows that overlap. In this article, we…

Working with Intersection in MySQL

Intro The INTERSECT operator allows us to join two queries together and only select the rows they share in common. This is similar to UNION except we only get rows that overlap. In this article, we…

Working with Union in Postgres

Intro If we have multiple select with results that we want to combine, we can us the UNION operator. Unlike a join, the UNION will stack results together. In this article, we will learn how to use…

Working with Union in MySQL

Intro If we have multiple select with results that we want to combine, we can us the UNION operator. Unlike a join, the UNION will stack results together. In this article, we will learn how to use…

Working with CTEs in Postgres

Intro A Common Table Expression or CTE is a temporary table result that exists within a single query. The provide better performance and readability compared to derived tables and can self reference…

Working with CTE in MySQL

Intro A Common Table Expression or CTE is a temporary table result that exists within a single query. The provide better performance and readability compared to derived tables and can self reference…

Working with Exists in Postgres

Intro The EXISTS operator is an operator that returns true or false. We can use this clause to check if there are an items in a subquery. In this article, we will learn how to use EXISTS in PostgreSQL…

Working with Exists in MySQL

Intro The EXISTS operator is an operator that returns true or false. We can use this clause to check if there are an items in a subquery. In this article, we will learn how to use EXISTS in MySQL. The…

Working with Derived Table in Postgres

Intro A derived table is a subquery used in the FROM clause. Often we want to select a subset of a table, then do further selection on that derived table. In this article, we will learn how to use…

Working with Derived Table in MySQL

Intro A derived table is a subquery used in the FROM clause. Often we want to select a subset of a table, then do further selection on that derived table. In this article, we will learn how to use…

Working with Subquery in Postgres

Intro A Subquery is a nested query inside a query. We can often combine to queries to make more advanced conditions. For example, we can search for orders that belong to a list of customers that meet…

Working with Subquery in MySQL

Intro A Subquery is a nested query inside a query. We can often combine to queries to make more advanced conditions. For example, we can search for orders that belong to a list of customers that meet…

Working with Having in Postgres

Intro The HAVING clause allows us to filter group rows from SQL. When using the GROUP BY clause, our groups are created from rows already filters, from using the WHERE clause. Thus, we can use the…

Working with Having in MySQL

Intro The HAVING clause allows us to filter group rows from SQL. When using the GROUP BY clause, our groups are created from rows already filters, from using the WHERE clause. Thus, we can use the…

Working with GroupBy in Postgres

Intro The Group By clause allows us to summarize multiple rows into a less or even a single row. For example, if we want to count the number of people with the same last name or sum the number of…

Working with GroupBy in MySQL

Intro The Group By clause allows us to summarize multiple rows into a less or even a single row. For example, if we want to count the number of people with the same last name or sum the number of…

Working with Joins in MySQL

Intro Often when working with SQL we have multiple tables that are related. For example, we have have a user and their preferences in two separate tables. We can combine the information in both tables…

Working with ALIAS in Postgres

Intro Aliases help you improve the readability of your queries. Sometimes, the column names are technically and you wish to change the names to be more human readable. Also, when joining table, you…

Working with Joins in Postgres

Intro Often when working with SQL we have multiple tables that are related. For example, we have have a user and their preferences in two separate tables. We can combine the information in both tables…

Working with ALIAS in MySQL

Intro The LIKE operator allows to filter values based on if a string contains a specified pattern or not. In this article, we will learn how to use the LIKE operator in MySql. The Syntax The basic…

Working with LIKE in Postgres

Intro The LIKE operator allows to filter values based on if a string contains a specified pattern or not. In this article, we will learn how to use the LIKE operator in PostgreSQL. The Syntax The…

Working with LIKE in MySQL

Intro The LIKE operator allows to filter values based on if a string contains a specified pattern or not. In this article, we will learn how to use the LIKE operator in MySql. The Syntax The basic…

Working with IS NULL in PostgreSQL

Intro PostgreSQL provides the operator to help test for NULL fields. Since NULL is a special value to represent an empty state, we have special operators to use this value. In this article, we will…

Working with IS NULL in MySQL

Intro MySQL provides the operator to help test for NULL fields. Since NULL is a special value to represent an empty state, we have special operators to use this value. In this article, we will learn…

Working with In in Postgres

Intro PostgreSQL provides the IN operator to match a list of values. Rather than writing a list of OR clauses we can use the IN as a shortcut. For example, say we have a list of cities and addresses…

Working with IN in MySQL

Intro MySql provides the IN operator to match a list of values. Rather than writing a list of OR clauses we can use the IN as a shortcut. For example, say we have a list of cities and addresses. We…

Working with Between in Postgres

Intro When viewing data, we often run into duplicate entries. For example, we may want to query addresses, and we should expect to see multiple addresses with the same country. If we would like to see…

Working with Distinct in MySQL

Intro When viewing data, we often run into duplicate entries. For example, we may want to query addresses, and we should expect to see multiple addresses with the same country. If we would like to see…

How to Create a Database PostgreSQL

Intro In PostgreSQL you will often start by creating and selecting which database to use. You need a database to store data, so this is a logically place to start. In this article, we will learn how…

How to Create a Database in MySQL

Intro In MySql you will often start by creating and selecting which database to use. You need a database to store data, so this is a logically place to start. In this article, we will learn how to…

Working with Between in Postgres

Intro The BETWEEN operator allows you to filter for values in between two other values. For example, we could select items that are in a specific date range or houses in between a price range. In this…

Getting Started with Laravel

Getting Composer To get started with laravel, we will first need which is a package manager for php. You can install it in a few ways. If on mac, we recommend homebrew: https://formulae.brew.sh…

Working with Between in MySQL

Intro The BETWEEN operator allows you to filter for values in between two other values. For example, we could select items that are in a specific date range or houses in between a price range. In this…

Working with Where in Postgres

Intro The WHERE cause allows you to filter results returned by the SELECT statement. For example, we can filter a list of actors by their first name. In this article, we will learn how to use the…

Working with Select in Postgres

Intro The SELECT clause is the main entry to sql. The command allows us to query data from our SQL database. In this article, we will learn how to use Postgres Select. Getting Setup For our setup, we…

Working with Orderby in Postgres

Intro When viewing rows for our data base, we often want to sort the data for some meaningful analysis. To do this, we can use the clause. In this article, we will learn how to use ORDER BY in…

Working with Limit in Postgres

Intro We often work with large databases and only want to return a small number of rows to view our data. Postgresql provides the LIMIT keyword to help with this. In this article, we will learn how to…

Working with Delete in Postgres

Intro When we want to remove rows, we can use the statement. We can remove all rows in a table, or use selectors like WHERE and LIMIT to remove less. In this article, we will learn how to use DELETE…

Getting Started with Postgres and Docker

Getting Setup For our setup, we will use docker compose to create a Postgres database and to connect phpmyadmin. Start by copying the following into a docker compose file called We can run this file…

Working with MySql Limit and Offset

Intro We often work with large databases and only want to return a small number of rows to view our data. Mysql provides the LIMIT keyword to help with this. In this article, we will learn how to use…

Getting Started with MySql and Docker

Intro Working with Databases has never been easier. With the use of image based systems like docker, we can quickly get a dev environment running without running through a lot of installs. In this…

Working with MySql Order By

Intro When viewing rows for our data base, we often want to sort the data for some meaningful analysis. To do this, we can use the clause. In this article, we will learn how to use ORDER BY in MySql…

Working with MySql Delete

Intro When we want to remove rows, we can use the statement. We can remove all rows in a table, or use selectors like WHERE and LIMIT to remove less. In this article, we will learn how to use DELETE…

Working with MySql Where

Intro The WHERE cause allows you to filter results returned by the SELECT statement. For example, we can filter a list of actors by their first name. In this article, we will learn how to use the…

Working with JSON in Go

Intro JSON is a domain specific language (DSL) that allows you to defined data. It is an alternative to YAML and is used in many apps to send responses via REST apis to config files. In this article…

Working with MySql Select

Intro The SELECT clause is the main entry to sql. The command allows us to query data from our SQL database. In this article, we will learn how to use MySql Select. Getting Setup We will be using…

Working with YAML in Go

Intro YAML is another domain specific language (DSL) that allows you to defined data. It is an alternative to JSON and is used heavily in apps such as Kubernetes. In this article, we will learn how to…

Working with Email in Go

Intro Go provides an smtp package which uses the SMTP standard to send email. Sending email is a common use in most programs. In practice, you will often want to use a service such as Mailgun and…

Working with Scope in Go

Intro Scope in programming languages defines the region where a variable exists. In Go, we have three places to declare variables, local, global, and formal parameters. In this article, we will learn…

Working with Type Conversion in Go

Intro Type conversion lets you easily change the types of your variables in Go. We can do this using the cast operator. In this article, we will learn how to use type conversion in Go. The Case…

Working with Logs in Go

Intro Logs allow you to send info or errors during your program for debugging purposes. In practice, logging is often used in conjunction with something like Elastic Search to monitor and track down…

Working with Channels in Go

Intro Channels allow you to shared data between goroutines. When executing multiple concurrent tasks, you often want to send and receive data from these tasks. Channels allow you to do this. In this…

Working with WaitGroups in Go

Intro When using goroutines, you often need to wait for many goroutines to complete. You can set a timer, but this can be inconsistent as you don't always know how long the goroutines will take. To…

Working with GoRoutines in Go

Intro Go routines allow you to execute operations or functions concurrently. These are Go's way of handling multi-threading or asynchronous programming. In this article, we will learn how to us…

Working with Panic in Go

Intro Go doesn't have an exception mechanism, instead we have panic. During run time, if there is an execution problem, Go will throw a panic. This will stop code execution, completed all deferred…

Working with Deferred Functions in Go

Intro Go doesn't have an exception mechanism, instead we have panic. During run time, if there is an execution problem, Go will throw a panic. This will stop code execution, completed all deferred…

Working with Variadic Functions in Go

Intro Variadic functions allow you to pass a variable number of arguments. If you are creating a function, but want to allow for various arguments, you can use a variadic function. For example. The…

Working with Switch in Go

Intro In go, the switch statement allows us to direct program flow in a slightly different way. Instead of multiple if else statements, we can use switch on a single expression. In this article, we…

Working with Loop Control in Go

Intro Interfaces in Go work a bit different than in other languages. First of all, you specify only the methods that an interface should conform to and not the properties. Also, interfaces are…

Working with Interfaces in Go

Intro Interfaces in Go work a bit different than in other languages. First of all, you specify only the methods that an interface should conform to and not the properties. Also, interfaces are…

Working with If Else in Go

Intro Go provides us with the common , , and control statements that you would expect from programming languages. These statements allow you to branch and run code based on conditions. In this…

Working with Struct in Go

Intro In Go, structs allow you to create your own data types by grouping multiple types together. For example, we can create a person struct to hold a first name, last name, and more. In this article…

Working with Range in Go

Intro The range keyword can be used with the for keyword to loop through Go data types. We can loop through arrays, slices, maps, and channels. In this article, we will learn how to use Go range…

Working with Maps in Go

Intro Maps allow us to build a map of keys to values. For example, we can map a list of users to phone numbers to quickly retrieve the information. In this article, we will learn how to use Maps in Go…

Working with Slices in Go

Intro Slices are super charged arrays. They allow you to dynamically build, add, and modify lists. Similar to arrays, they must have the same site, but they don't require a static size. In this…

Working with Arrays in Go

Intro Arrays are a fixed list of variables all with the same size. For example, we can have an array of 3 integers, or 4 strings, but we cannot mix them. Because arrays are fixed, their size cannot…

Working with strings in Go

Intro In Go, strings are a read-only sequences of bytes. If you know other programming languages, you are probably used to managing a sequence of characters. Go uses bytes to support many characters…

Working with Functions in Go

Intro Function allow you to group code together and easily reuse code. In Go, we always have a main function where the program starts. From then on, we can build out functions to keep our code…

Working with For Loops in Go

Intro Loops allows you to easily repeat code by telling the compiler how many times to execute a block of code. In this article, we will learn how to use loops in Go. Loops in Go Go is slightly…

PySpark Binarizer

Intro The PySpark Binarizer allows you to convert a continuous variable into a discrete 0 or 1. This is helpful when you want to simply convert a column to check if a value exists based on a threshold…

PySpark Statistics Package

Intro PySpark provides a stats library located in that gives us a few tests and classes to use to do common statistical flows. The are based on vectors and provide scalable operations. In this…

PySpark Vectors

Intro PySpark provides several methods for working with linear algebra methods in the machine learning library. Specifically, we have a few ways to build and work with vectors at scale. In this…

PySpark Difference Between Two Dates

Intro PySpark provides us with and that allows us to get the time differences between two dates. This is helpful when wanting to calculate the age of observations or time since an event occurred. In…

PySpark Date_Format

Intro The PySpark function allows use to convert date columns into string columns using a specific output. A common task would be to convert to and iso8601 standard for exporting to other systems. In…

PySpark To_Date

Intro The PySpark method allows us to convert timestamps to a date type. This is useful if you import csv data that has date strings, but you want to perform date options on it. In this article, we…

PySpark Substring

Intro The PySpark method allows us to extract a substring from a column in a DataFrame. In this article, we will learn how to use substring in PySpark. Setting Up The quickest way to get started…

Working with Operators in Go

Intro Operators allow us to perform mathematics, comparisons, and much more in programming. Go provides a great set of operators for your daily use. In this article, we will learn how to use operators…

Working with Constants in Go

Intro Constants are variables that have a fixed value and cannot be changed. They are often use to keep static values that you don't want changed later on. For example, you may have a size for a cache…

C# Type Conversion

Intro Type conversion or type casting is a common feature in programming levels that allows you to go from one type to another. This is common when converting types to strings for logging or…

PySpark ConcatWS

Intro The PySpark function allows you to concatenate an array field into a single Sting field. This serves as the opposite of the function. This allows you to perform string operations on a column…

PySpark Split

Intro The PySpark method allows us to split a column that contains a string by a delimiter. For example, we have a column that combines a date string, we can split this string into an Array Column…

PySpark Fillna

Intro The PySpark and methods allow you to replace empty or null values in your dataframes. This helps when you need to run your data through algorithms or plotting that does not allow for empty…

Working with Variables in Go

Intro Declaring variables is a fundamental task in all programming languages. To do this in Go, we can use the following formula: is a reserved keyword that tells go we are going to declare a…

PySpark ForEach

Intro The PySpark method allows us to take small samples from large data sets. This allows us to analyze datasets that are too large to review completely. Setting Up The quickest way to get started…

PySpark ForEach

Intro The PySpark method allows us to iterate over the rows in a DataFrame. Unlike methods like map and flatMap, the method does not transform or returna any values. In this article, we will learn…

PySpark FlatMap

Intro The PySpark method allows use to iterate over rows in an RDD and transform each item. This method is similar to method, but will produce a flat list or array of data instead of mapping to new…

PySpark Map

Intro The PySpark method allows use to iterate over rows in an RDD and transform each item. Mapping is a common functional operation and PySpark allows us to use this at scale. In this article, we…

PySpark UDF (User Defined Function)

Intro Similar to most SQL database such as Postgres, MySQL and SQL server, PySpark allows for user defined functions on its scalable platform. These functions can be run on dataframes or registers to…

PySpark UnionByName

Intro When merging two dataframes with , we sometimes have a different order of columns, or sometimes, we have one dataframe missing columns. In these cases, PySpark provides us with the method. In…

PySpark Union

Intro PySpark provides us with the function to merge two or more data frames together. There also exists a method that was deprecated since Spark 2.0, but can be used if you have an older version…

PySpark GroupBy

Intro When working we often want to group data to view distributions or aggregations. PySpark provides us with the method to group our dataframes. In this particle, we will learn how to work with…

PySpark Drop OrderBy

Intro When working with data and viewing, we often want to sort or order our data for easier review. PySpark provides the and functions to sort dataframes. In this article, we will learn how to use…

PySpark Drop Duplicates

Intro During the data cleaning process, we would like to remove duplicate rows. PySpark provides us with the and that let's us remove duplicates on large amounts of data. In this article, we will…

PySpark Filter

Intro Often when working with dataframes we want to filter our data to a subset. PySpark provides us with the and the alias to filter our data frames. In this article, we will learn how to use…

PySpark WithColumnRenamed

Intro The allows us to easily change the column names in our PySpark dataframes. In this article, we will learn how to change column names with PySpark withColumnRenamed. Setting Up The quickest way…

PySpark WithColumn

Intro The method allow us to add columns, modify their types, modify their values and more. It is one of the most commonly used methods for PySpark. In this article, we will learn how to use PySpark…

PySpark Collect

Intro The dataframe collect method is used to return the rows in a dataframe as a list of PySpark Row classes. This is used to retrieve data on small dataframes so that you can inspect and iterate…

PySpark Select

Intro Selecting columns is one of the most common operations when working with dataframes. We can select by position or name. We can also select a single or multiple columns. In this article, we will…

PySpark Row

Intro The PySpark Row class is located in the module and provides a simple way to create rows or observations in a dataframe or an RDD. In this article, we will learn how to use PySpark Row. Let's…

PySpark StructType and StructField

Intro PySpark provides two major classes, and several other minor classes, to help defined schemas. This allows us to interact with Spark's distributed environment in a type safe way. In this article…

PySpark Show

Intro The show function allows us to preview a data frame. The show method provides us with a few options to edit the output. In this article, we will learn how to use the PySpark show function. We…

PySpark Window Functions

Intro Computing operations over a window of data, or a subset, is a common task. Often we want to rank information or subsets of data. For example, we may want to see the top sales per each month. In…

PySpark Pivot (rows to columns)

Intro Often when viewing data, we have it stored in an observation format. Sometimes, we would like to turn a category feature into columns. We can use the Pivot method for this. In this article, we…

PySpark DataFrame Join

Intro Often you will have multiple datasets, tables, or dataframes that you would like to combine. For example, you may have customers and their purchases and would like to see these in a single…

PySpark DataFrame Aggregations

Intro One main feature you will use in Spark is aggregation. This will help with exploratory data analysis and building dashboards that scale. In this article, we will learn how to use pyspark…

PySpark DataFrame Select, Filter, Where

Intro Filtering and subsetting your data is a common task in Data Science. Thanks to spark, we can do similar operation to sql and pandas at scale. In this article, we will learn how to use pyspark…

PySpark Handle Null Data

Intro Often when working with data you will find null values. It is a common task to work with and know how to manage these null values. The decision to drop or to impute is important in the model…

PySpark Create Dataframe

Intro There are many ways to create a data frame in spark. You can supply the data yourself, use a pandas data frame, or read from a number of sources such as a database or even a Kafka stream. In…

Redis Sort with Node

Intro Sorting is a common programming task, and you may be tempted to pull data from Redis and sort it client side. However, using Redis's built in sort function will be more performant and general…

Redis Sort with Python

Intro Sorting is a common programming task, and you may be tempted to pull data from Redis and sort it client side. However, using Redis's built in sort function will be more performant and general…

Redis Server Sessions in Python

Intro Sessions are usually short lived data, or at least have an expiration date, used to transfer state accross RESTful applications. REST applications are stateless per their spec, yet sometimes we…

Backing Up and Restoring Redis

Intro Redis uses two methods for persistence, snapshotting and append only file. Both have different use cases and can be used separately or in conjunction. In this article, we will learn how to…

Redis Server Sessions in Node

Intro Sessions are usually short lived data, or at least have an expiration date, used to transfer state accross RESTful applications. REST applications are stateless per their spec, yet sometimes we…

Redis Transactions in Node

Intro Transactions are a common database requirement for when you need to make multiple insert or update operations together. That is, if one operation fails, you don't want to execute either…