Notes from “Programming Elixir”

Introduction

In some cases, Groovy scripts are loaded (by script engine of the hosted application server) as a single piece and have no dependency to other shared libraries. In this situation, the scripts need to be self-contained with both application code and support code. How to split these two parts of code and how to make use of Groovy’s dynamic language features is the main topic of this blog. There are also some suggestions of what is good code style about Groovy (for Java programmers).

It’s a summary and abstraction of “Design Data-intensive Applications” (DDIA), which is an excellent book covering a vast majority domain of database administration and distribute systems.

Intermediate Level

1
2
3
4
5
6
7
8
9

# 01
Name = collections.namedtuple('Name', ['first','last'])
aName = Name('John', 'Doe')
# first = John, last = Doe

# 05
def score2grade(score, ranges=[60,70,80,90],grades=[E,D,C,B,A]):
	i = bisect.bisect(ranges,score)
	return grades[i]

reference

Fluent Python 2015.7

Tips of Anaconda installation

For Anaconda distribution, this (mirror)[https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/] could be faster than the official.
(numpy tutorial)[https://docs.scipy.org/doc/numpy-dev/user/quickstart.html]

matrix with Jupyter

1
2
3
4
5
6
7
8
9

%matplotlib inline
import sympy as sympy
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sbn
from scipy import *

A = np.array(([.8,.3],[.2,.7]))
A.dot(x1)

Language Basic

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

// 11
def foo(**user_info):
    res = {}
    for k, v in user_info.items():
        res[k] = v
    return res


user = foo(lastName="Foo", firstName="Bar")
for k, v in user.items():
    print("{} -> {}".format(k, v))

// 13
def foo(
        param1,
        param2,
        param3):
    // function body

// 23
names = [("Tony","Zhang"),("Anri","Liu"),("Charry","Y")]
fns, lns = zip(*names)
fns // ("Tony","Anri","Charry")
lns // ("Zhang","Liu","Y")

reference

scikit

Multithreading, Parallel and Async

Multithreading

A form of concurrency that uses multiple threads of execution.

Thread, BackgroundWorker, low-level, best to start with high-level abstractions like thread pool.

Parallel

Divide work among multiple threads that run concurrently.

Maximize the use of multiple processors of CPU. In most cases, work against built-in parallelism on server-side.

Async

Use futures or callbacks to avoid unnecessary threads.

For many programmers of younger generation, career started with object-oriented programming. And modern languages provide easy ways to create objects (and destroy them after the use). As a result, programs are pushed to the limit to make use of encapsulation and polymorphism and therefore sometimes tend to be unnecessarily complicated and hard to maintain. Let’s use an example to illustrate when and when not to (over)use object-oriented programming.

What is the problem

Continue with the last topic about domain model, there is a performance problem. See the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

public Supplier<List<Person>> femaleMembers() {

  return () -> {

    List<ID> idOfAdmins = loadGroup.apply(groupName).admin.stream()
      .map(admin -> admin.id);

    Predicate<ID> isAdmin = id -> 
      idOfAdmins.stream().anyMatch(adminId -> adminId != id); 

    List<ID> idOfOwners = getMails.apply(groupName).stream()
      .map(mail -> mail.idOfOwner.orElse(null))
      .filter(id -> isAdmin.andThen(ID::isNotEmpty).test(id)));
  
    return getPersons.apply(idOfOwners).stream()
      .filter(person -> person.gender == FEMALE); 
  };
}

Given the same groupName, if femaleMembers is invoked by more than one place within the same transaction, it will trigger the query each time even when the result is almost identical. So how to improve the performance?

What is the problem

Suppose we have domain models like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

class Person {
  ID id;
  PersonName name;
  Gender gender;
}

class Mail {
  Address address;
  MailGroup group;
  ID? idOfOwner;
}

class MailGroup {
  String name;
  Person[] admin;
}

One day, product owner asked me to do a story with the following requirement:

Given a mail group, list female owners of mail addresses in this group. The list should exclude the admin of this mail group for whatever reason.

What is the problem

Suppose we have a Person model like this:

1
2
3
4
5
6
7
8

class Person {
	private ID id;
	private PersonName name;
	private BirthDate dateOfBirth;
	private TransactionalDate lastModified;
	private List<Contact> contacts;
	...
}

Something additional about the way of modeling. I could create the same Person in this way:

1
2
3
4
5
6
7

class Person {
	private long id;
	private String name;
	private Date dateOfBirth;
	private Date lastModified;
	...
}

The benefit of using ID, PersonName and BirthDate instead of long, String and Date respectively is that:

1. These basic types could be very likely reused in other models. Having a separate definition gives them a place to hold their own business logic other than the generic type shipped with language library. For example, ID is a numeric long greater than zero (due to the fact of incremental primary key in database). ID could have a property ‘Empty’ to indicate the data has not been persisted yet, which is better than using expression if (id > 0) that leaks too much technical details. And for BirthDate, we can apply a constraint on it due to the limit of human being’s age; while for TransactionalDate, it could have a default value as the current date, which represents the timing of a business transaction.
2. Less possibility of using a wrong field because most fields have a different type so that compiler can easily detect such error. That’s where the misleading practice, ‘Don’t use the same type for different fields when modeling a business object’, comes from.

And in application code we need to send person data to an external service like this:

1
2

externalService.consumePerson(long personID, String name, Date birthDate, ...);
externalService.consumePersonAndContact(long personID, String name, List<ContactInfo> contacts, ...);

Note external service has a slightly different understanding of person data. What’s more, different services ask for different data. How to do this with clean code?