DevTalks brings seasoned developers around Helsinki together to listen to industry-recognized speakers present intriguing ideas, techniques and technologies. Last week at the fifth edition of DevTalks, we were honored to host a great crowd of 100+ software developers and two fantastic speakers: Talia Nassi and Gabriel Lebec.
Who were speaking?
The first speaker of the evening was Talia Nassi, a quality-driven Test Engineer at WeWork. She has a passion for breaking and rebuilding software to the highest possible quality. Talia started as a QA intern while studying at UC San Diego and immediately knew that she had found her calling. From UCSD she was recruited to work at Visa, where she tested the payment processing system for the Prepaid Cards. After Visa, Talia started at WeWork, where she continues to do what she loves—deliver high quality software!
After Talia, Gabriel Lebec took the stage. Gabriel is a software engineer who develops Search features at Google. As a former instructor at Fullstack Academy of Code, he has helped hundreds of students launch careers in technology. He holds a BA in Mathematics and Studio Art from Georgetown University, and is an enthusiast of functional programming in Haskell.
Talia Nassi: Testing in Production
Testing in production sounds scary—isn't production too late to detect a defect as it would already be visible to customers? Talia didn't seem to think so. She first stumbled upon the idea when she interviewed for a job. When she asked the interviewers about their testing practices, more specifically, how they keep their staging environment up to date, she was perplexed by their answer. "We don't have a staging environment, we test in production". I can imagine Talia being as apprehensive about these testing practices back then as many of us felt in the audience.
Indeed our concerns aren't entirely unfounded. Testing in production can affect real users and introduce noise into analytics that inform critical business decisions. It can also affect third parties integrated into the software, like send unnecessary API calls, or even corrupt production data.
But on the other hand, the idea of a staging environment comes with significant problems of its own. First, staging test results don’t always match production results because staging and production data are different. Similarly, staging traffic and load patterns don’t usually match production, making it hard to emulate production performance characteristics. Other critical factors that speak against testing in staging environments are that maintaining an up-to-date staging environment is expensive, and those staging environments aren't usually as heavily monitored as production.
Given the differences between typical staging and production environments, Talia believes the only way to be entirely sure that a feature works reliably is to test it in production. However, many companies decide not to do it out of fear or because they don't trust their tooling, products, or processes.
Safe production testing requires a few things to be in place:
- Use feature flags to hide newly introduced features from the public. At Smartly.io, we use feature flags extensively and have built tools around them to allow customer success managers to enable specific features for individual early-bird customers. The same infrastructure lets us perform internal rollouts to gather feedback before public releases.
- Introduce a test automation framework into the development process to speed up the feedback loop. Run the tests in production regularly, like at fixed intervals or on commit.
- Introduce an alert tool so that relevant people get notified if something goes wrong. At Smartly.io, we use Bugsnag for frontend error alerts.
- Create and manipulate test data in production. Use consistent naming conventions for test users. Test users must look and act like real users, but be distinguishable by their name (or another attribute) to the reporting systems. Also, ensure test users only interact with other test users in automation scripts.
- Write your tests (BDD is great!) and ensure clean setup and teardown. Trigger the alerting system on setup and teardown errors.
- Do a production canary rollout.
At Smartly.io, we have tooling in place to test changes against production. All our micro-frontends are linked to the main (host) frontend using tools like Yalc. They can then easily be launched using a webpack server against the production backend with a single command.
A handy command for running the main Smartly frontend code in interactive mode and connected to our production backend.
What if you can't test in production (like for highly sensitive medical systems)? In that case, consider simulating the production environment with container technologies. When factors like privacy are considered, this makes sense.
You’ll need the buy-in of your colleagues and company’s technology decision makers to shift to testing in production..To succeed, you’ll need to map out the pros and cons of production testing, cite examples from the past where production testing could have mitigated issues, and present a sound implementation plan for the new testing strategy. Always keep in mind that “it works on my computer” doesn’t cut it, nor does “it works on the CI server”; it only really works if it works reliably in production.
There has been a recent resurgence in functional programming patterns with many new languages implementing first-class functions, frameworks introducing functional components and libraries encouraging the use of immutable data structures. This resurgence is due to the bad taste that bugs from the shared mutable state have left in many mouths, as well as the inconvenience that comes with testing and debugging large stateful software components.
Quick introduction to lambda calculus
A lambda is a function that takes one argument and returns an expression. For example, the Identity Function takes an argument and returns that argument. An anonymous function, together with its argument and returned expression are sometimes called a lambda abstraction.
According to Wikipedia, “Lambda calculus (λ-calculus) is a formal system in mathematical logic for expressing computation based on function abstraction and application using variable binding and substitution. It is a universal model of computation that can be used to simulate any Turing machine.”
Important notes to keep in mind about lambda calculus:
- In lambda calculus, there’s no such thing as a mutable variable. Instead, all variables are immutable. Variables have a definition (known or unknown) but they are never reassigned.
- Lambdas take only one argument. An expression such as
a => b => c => b.
- Lambdas can be nested, which means the innermost is evaluated then passed as an argument to its outer lambda and so forth.
- Function evaluation means symbol replacement. Symbols in the expression are simplified and replaced “recursively”. This is also called “β-replacement”.
- An expression is said to be in “β-normal” form if no further β-replacement is possible.
Example 1: the Identity function
This is a function that takes an argument and returns the argument, let's denote it by
|I := λa.a||I = a => a|
Therefore the Identity of 1 denoted as
I(1) == 1.
I(2) == 2.
I(I) == I (the identity of the identity function is the identity function itself).
Example 2: the Kestrel function
The Kestrel function “takes” two arguments “a” and “b” then returns “a”. Remember that in lambda calculus functions only take one argument, so whenever we say a function takes more than one argument, the others are curried in.
|K := λab.a||K = a => b => a|
So it follows that
K(I)(X) === I. Kestrel takes two arguments, in this case I and X, and returns the first argument, in this case I.
K(I)(X)(y)? As shown above,
K(I)(X) === I, therefore
(y) === I(y) === y.
Let’s pay a bit more attention to this conclusion:
K(I)(X)(y) === y. The expression K(I) takes two arguments X and y and returns the second argument. Now we can see how these fundamental functions, even though trivial by themselves, form building blocks which could be combined to build more complicated machinery.
K takes two arguments and returns the first, while KI takes two arguments and returns the second. The expression KI is called the Kite function. There are several other fundamental functions like these, you can find the definitions of some these bird-named functions in this F# discussion.
Why care about these functions?
Common boolean operations and their Church encodings (lambda representation) from Gabriel's presentation
DevTalks will be back!
The fifth DevTalks was both entertaining and inspiring. I would like to thank Talia and Gabriel for the talks and all the participants who asked insightful questions and contributed to intriguing conversations after the talks. DevTalks will be back next spring. You can stay up to date by joining our Meetup.com group.
Tune into DevTalks with the Youtube playlist featuring Talia and Gabriel’s talks, plus the official aftermovie👇
Feel free to subscribe for more brainfood for engineers, by engineers.