Browse Source

Merge tag '20190703.1' into preview

20190703.1
master
Julio Biason 5 years ago
parent
commit
e0445f02a2
  1. 10
      content/books/things-i-learnt/cognitive-cost/index.md
  2. 16
      content/books/things-i-learnt/disclaimer/index.md
  3. 16
      content/books/things-i-learnt/functional-programming/index.md
  4. 24
      content/books/things-i-learnt/gherkin/index.md
  5. 60
      content/books/things-i-learnt/integration-tests/index.md
  6. 28
      content/books/things-i-learnt/intro/index.md
  7. 33
      content/books/things-i-learnt/magical-number-seven/index.md
  8. 10
      content/books/things-i-learnt/patterns-not-solutions/index.md
  9. 5
      content/books/things-i-learnt/spec-first/index.md
  10. 11
      content/books/things-i-learnt/steps-as-comments/index.md
  11. 34
      content/books/things-i-learnt/tests-dead-code/index.md

10
content/books/things-i-learnt/cognitive-cost/index.md

@ -36,7 +36,7 @@ sum(is_pred(x) for x in my_list)
```
Wait, didn't I say that `sum()` sums numbers? And that `is_pred()` returns a
boolean? How can I sum booleans? What's the expected result of True + True +
boolean. How can I sum booleans? What's the expected result of True + True +
False?
Sadly, this works. Because someone, long time ago, didn't think booleans were
@ -47,11 +47,11 @@ But, for you, you'll now read a line that says "summing a boolean list returns
a number". And that's two different, disparate things that you suddenly have
to keep in mind when reading that line.
That's why [types are important](/books/things-i-learnt/data-types) are
important. Also, this may sound a bit like [the magical number
That's why [types are important](/books/things-i-learnt/data-types). Also,
this may sound a bit like [the magical number
seven](/books/things-i-learnt/magical-number-seven), 'cause you have to keep
two things at your mind at the same thing but, although that's not near seven,
they are not the same, with opposite (for weird meanings of "opposite", in this
case) meanings.
they are not the same, with opposite (for weird meanings of "opposite", in
this case) meanings.
{{ chapters(prev_chapter_link="/books/things-i-learnt/magical-number-seven", prev_chapter_title="The Magic Number Seven, Plus Or Minus Two", next_chapter_link="/books/things-i-learnt/functional-programming", next_chapter_title="Learn The Basics of Functional Programming") }}

16
content/books/things-i-learnt/disclaimer/index.md

@ -14,10 +14,11 @@ personal opinion
A lot of stuff I'm going to discuss throughout this book will come directly
from my personal experience in several projects -- system applications, web
backend, embedded, mobile, stream processing -- in several different languages
-- C, C++, Python, Java. And, because it comes from personal experience,
everything reflects my own personal opinion on several subjects.
-- C, C++, Python, Java, Clojure, Rust. And, because it comes from personal
experience, everything reflects my own personal opinion on several subjects.
Obviously, you don't need to agree with every single point.
Obviously, you don't need to agree with every single point. But I hope at
least it will make you rethink a few subjects.
Also, sometimes I may mention some examples that people who know me -- either
worked with me, heard me complain about some project, inherit one of my
@ -32,10 +33,11 @@ things properly in a crunchtime. And that's why some things don't look as
pretty as they should. Heck, if you think I'm attacking the original author of
some example, look back the stuff I wrote and you'll see things a lot worse.
But I need the example. I want to show people how things can be better. I want
to show people how my opinion built over some subject. And, again, I'm in no
way attacking the original author of the code. I may even call the code
"stupid", but I'm not calling the author _stupid_.
But I need the example. I have this hope that showing people a few mistakes
can make things better. I want to show people how my opinion built over
some subject. And, again, I'm in no way attacking the original author of the
code. I may even call the code "stupid", but I'm not calling the author
_stupid_.
With that in mind...

16
content/books/things-i-learnt/functional-programming/index.md

@ -6,7 +6,7 @@ date = 2019-06-26
tags = ["en-au", "books", "things i learnt", "functional programming"]
+++
At this point, you should at least have hard about how cool functional
At this point, you should at least have heard about how cool functional
programming is. There are a lot of concepts here, but at least the very basic
ones you should keep in mind.
@ -18,14 +18,14 @@ A lot of talks about functional programming come with weird words like
programming is actually easy to understand and grasp.
For example, immutability. This means that all your data can't change once
it's created. You have a record with user information and the user changed
this password? No, do not change the password field, create a new user record
with the updated password and discard the old one. Sure, it creates a lot of
create and destroy sequences which makes absolutely no sense (why would you
it's created. Do you have a record with user information and the user changed
their password? No, do not change the password field, create a new user record
with the updated password and discard the old one. Sure, it does a lot of
"create and destroy" sequences which makes absolutely no sense (why would you
allocate memory for a new user, copy everything from the old one to the new
one, update one field, and deallocate the memory from the old one? It makes no
sense!) but, in the long run, it would prevent weird results, specially when
you understand and start use threads.
one, update one field, and "deallocate" the memory from the old one? It makes
no sense!) but, in the long run, it would prevent weird results, specially
when you understand and start use threads.
(Basically, you're avoiding a shared state -- the memory -- between parts of
your code.)

24
content/books/things-i-learnt/gherkin/index.md

@ -6,8 +6,8 @@ date = 2019-06-19
tags = ["en-au", "book", "things i learnt", "gherkin", "expectations"]
+++
Gherkin is file format for writing behaviour tests. But it can also give you
some insights on what you should do.
Gherkin is file format for writing behaviour tests (BDD). But it can also give
you some insights on what you should do.
<!-- more -->
@ -16,8 +16,9 @@ Alright, let's talk a bit about Gherkin:
[Gherkin](https://en.wikipedia.org/wiki/Cucumber_(software)#Gherkin_language)
is a file format created for [Cucumber](https://en.wikipedia.org/wiki/Cucumber_(software)),
which describes scenarios, what's in them, what actions the user/system will
do and what's expected after those actions, in a very high level, so people
without programming experience can describe what's expected from the system.
do and what's expected after those actions, in a very high level, allowing
people without programming experience can describe what's expected from the
system.
Although Gherkin was born with Cucumber, it is now supported by a bunch of
programming languages, through external libraries.
@ -30,7 +31,7 @@ A typical Gherkin file may look something like this:
Or, in a more concrete example:
* **Given that** The system is retrieving all tweets favourited by the user
* **Given that** The system is retrieving all tweets liked by the user
* **When** It finds a tweet with an attachment
* **Then** The attachment should be saved along the tweet text
@ -39,14 +40,15 @@ Pretty simple, right?
Now, why I'm mentioning this?
Sometimes, specs are not the most clear source of information about what it is
expected from the system. If you're confused about what you should write,
asking the person responsible for the request to write something like Gherkin
may give you some better insights about it.
expected from the system, and up can't think of [steps to do
so](/books/things-i-learnt/steps-as-comments). If you're confused about what
you should write, asking the person responsible for the request to write
something like Gherkin may give you some better insights about it.
Obviously, it won't be complete. People tend to forget the error situations --
people entering just numbers on names, letter in age fields, tweets with no
text and just attachments -- but at least with a Gherkin description of the
system, you can get a better picture of the whole.
like filling the name field with numbers, using characters in age fields,
tweets with no text and just attachments -- but at least with a Gherkin
description of the system, you can get a better picture of the whole.
Also, you may not like to write specs. That's alright, you can replace them
with Gherkin anyway.

60
content/books/things-i-learnt/integration-tests/index.md

@ -13,8 +13,8 @@ tests for the whole compared to tests of single things.
First, I just don't want to into a discussion about what's the "unit in a unit
test"[^1], so let's take the point that a unit test is a test that tests a
class/function, not the whole system, which would require data flowing through
several classes/functions.
class/function, not the whole system from end to end, which would require data
flowing through several classes/functions.
There are several libraries/frameworks that actually split this in a way that
you can't test the whole.
@ -22,28 +22,30 @@ you can't test the whole.
those combinations -- and one that I worked with. Due the bean container of
Java, the extensive use of Beans by Spring and the way Mockito interacts with
the container, it's pretty easy to write tests that involve only one class:
You can ask Mockito to mock every dependency injection in one class and mock
every injected class, simply using annotations.
You can ask Mockito to mock every dependency injection (so it injects mocked
beans instead of the real ones) in one class and mock every injected class,
simply using annotations.
And this is cool and all. But the fact that we are making sure each class does
what it does, it doesn't give a proper view of the whole; you can't see if
that collection of perfectly tested classes actually solve the problem the
system is responsible for solving.
And this is cool and all and makes tests simple and fast. But the fact that we
are making sure each class does what it should do, it doesn't give a proper
view of the whole; you can't see if that collection of perfectly tested
classes actually solve the problem the system is responsible for solving.
Once, in C++, I wrote an alarm system
[daemon](https://en.wikipedia.org/wiki/Daemon_(computing)) for switches. There
were three different levels of things the alarm system should do: It could
only log the message of the incoming error, it could log the error and send a
SNMP message, or it could log the error, send a SNMP message and turn a LED in
the front panel on. Because each piece had a well defined functionality, we
broke the system in three different parts: One for the log, one for the SNMP
and one for the LED. All tested, all pretty. But I still had a nagging
feeling that something was missing. That's when I wrote a test that would
bring the daemon up, send some alarms and see the results.
were three different levels of things the alarm system should do, depending on
the incoming message from a service: It could only log the message of the
incoming error, it could log the error and send a SNMP message, or it could
log the error, send a SNMP message and turn a LED in the front panel on.
Because each piece had a well defined functionality, we broke the system in
three different parts: One for the log, one for the SNMP and one for the LED.
All tested, all pretty. But I still had a nagging feeling that something was
missing. That's when I wrote a test that would bring the daemon up, send some
alarms and see the results.
And, although each module was well tested, we still got one things we were
doing it wrong. If we never wrote an integration test, we would never catch
that.
those.
Not only that, but because we wrote a test that interacted with the daemon, we
could get a better picture of its functionality and the test actually _made
@ -51,19 +53,23 @@ sense_ -- as in, if you read the unit tests, they seemed disconnected from
what the daemon was expected to do, but the integration tests actually read
like "Here, let me show that we actually did what you asked". And yes, this
was akin to [Gherkin](/books/things-i-learnt/gherkin) tests, although I didn't
know Gherkin at the time.
know Gherkin at the time -- and, better yet, we had tests that proved that we
were following the [spec](/books/things-i-learnt/spec-first).
Personally, I think over time integration tests are more important that unit
tests. The reason is that I still have the feeling that unit tests check if
the classes/functions have _adherence_ to the underlying _design_ -- Does your
view can actually work without the controller? Is the controller using
something from the model or using things that should be in the view? -- but
adherence to the design gets better over time -- developers start using the
layout from previous examples, so they capture the design by osmosis, while
the big picture starts to get more and more complex, with lots of moving
parts.
Personally, I think over time integration tests become more important than
unit tests. The reason is that I personally have the feeling[^2] that unit
tests check if the classes/functions have _adherence_ to the underlying
_design_ -- Does your view can actually work without the controller? Is the
controller using something from the model or using things that should be in
the view? -- but adherence to the design gets better over time -- developers
start using the layout from previous examples, so they capture the design by
osmosis, while the big picture starts to get more and more complex, with lots
of moving parts.
[^1]: There is no "unit" in "unit tests". "Unit test" means the test _is_ a
unit, indivisible and dependent only on itself.
[^2]: Again, it's pure feeling from my experience. I have no data to back that
affirmation up, so take it with a grain of salt.
{{ chapters(prev_chapter_link="/books/things-i-learnt/functional-programming", prev_chapter_title="Learn The Basics of Functional Programming", next_chapter_title="Testing Every Function Creates Dead Code", next_chapter_link="/books/things-i-learnt/tests-dead-code") }}

28
content/books/things-i-learnt/intro/index.md

@ -7,29 +7,37 @@ tags = ["en-au", "books", "things i learnt", "intro"]
+++
"Things I Learnt The Hard Way (In 30 Years of Software Development)" started
as a simple sequence of toots (the same as "tweets", but outside Twitter) when
I was thinking about a new presentation I could do.
as a simple sequence of toots (the same as "tweets", on
[Mastodon](https://functional.cafe/@juliobiason) when I was thinking about a
new presentation I could do.
But why "a new presentation"?
<!-- more -->
I go around my state with a group called "Tchelinux": We usually go to
universities and talk to people starting uni, explaining things about
free/libre software and sometimes telling people about things they wouldn't
normally see in the uni curriculum.
I go around my state with a group called
"[Tchelinux](https://tchelinux.org/)": We usually go to universities and talk
to people starting uni, explaining things about free/libre software and
sometimes telling people about things they wouldn't normally see in the uni
curriculum.
One thing that annoys me is that there are very few presentations about "when
things go wrong". All the presentations are either prototypes or tell the good
things go wrong". All the presentations show prototypes or tell the good
stuff, and hide all the wrong things that could happen[^1]. Obviously, after
working 30 years in the field of software development, I saw my fair share of
things going wrong -- sometimes in unimaginable piles of crap -- and I thought
"maybe that's something people would like to hear".
(And, to be completely honest, some of those piles of crap were my own fault.)
And that's when the toot sequence started. Just before I noticed, I spent the
whole day just posting this kind of stuff (fortunately, my pile of "incoming"
was a bit empty at the time) and it had 30 points, plus addendums and a few
explanation points. That's when I decided to group all them in a single post.
whole day just posting this kind of stuff (fortunately, my pile of things in
the "incoming" folder was a bit empty at the time) and it had 30 points, plus
addenda and a few explanation points. That's when I decided to group all
them in a single post.
(Actually, I'm lying: Someone mentioned on Functional Café that I should make
a blog post for making it easier to read.)
All I thought when I grouped everything in a post was "this will make things
easier for the people following the thread on Mastodon". But then the post

33
content/books/things-i-learnt/magical-number-seven/index.md

@ -13,10 +13,14 @@ at the same time.
<!-- more -->
I've seen twice this weird construction on where a function would do some
processing, but its return value was the return of a second function and
some bit of processing. Nothing major. But the second function would also do
some processing and call a third function. And the third function would call a
fourth. And the fourth a fifth. And the fifth, a sixth function.
processing, but its return value was the return of this processing, plus the
result of a second function and some bit of processing. Nothing major. But the
second function would also do some processing and call a third function. And
the third function would call a fourth. And the fourth a fifth. And the fifth,
a sixth function.
And the "processing" was not something small, like "add two" or "multiply by
itself or a configurable value".
Something like this
@ -29,6 +33,12 @@ func_1
+-- func6
```
(If you need the _real_ processing I saw, it was a class that had a function
with some processing and then it would call a function in an injected
dependency -- which is pretty nice and dandy. But the injected dependency had
an injected dependency, and the third injected dependency _also_ had an
injected dependency, and so forth).
Now, when you're trying to understand this kind of code to find a problem,
you'll have to keep in mind what the first, second, third, fourth, fifth and
sixth functions do, 'cause they are all calling each other (inside them).
@ -36,8 +46,8 @@ sixth functions do, 'cause they are all calling each other (inside them).
This causes some serious mental overflow that shouldn't be necessary.
Not only that, but imagine that you put a log before and after `func_1`: The
log before points the data that's being send to func_1, and the log after its
result.
log before points the data that's being send to `func_1`, and the log after
its result.
So you'd end up with the impression that `func_1` does a lot of stuff, when it
actually is passing the transformation along.
@ -65,7 +75,14 @@ result6 = func_6(result5)
result7 = func_7(result6)
```
Now you can see _exactly_ how the data is being transfomed -- and, obviously,
(If we go back to the dependency injection chain, you may think that instead
of making DI7 receive DI6 as dependency [which would receive DI5 as
dependency, which would receive DI4 as dependency, which would receive DI3 as
dependency and so forth] you would actually create all the DI (dependency
injections) in one single pass and then the starting function would call the
dependencies in a single shot, not chaining them.)
Now you can see _exactly_ how the data is being transformed -- and, obviously,
the functions would have better names, like `expand`, `break_lines`,
`name_fields` and so on, so you can see that that compressed data I mentioned
before is actually being decompressed, the content is being broke line by
@ -77,7 +94,7 @@ add this additional step).
"But that isn't performant!" someone may cry. Well, maybe it's just a bit less
performant than the original chained-calls ('cause it wouldn't create and
destroy frames in the stack, it would just pile them up and then unstack them
destroy frames in the stack, it would just pile them up and then "unstack" them
all in the end), but heck, optimization is for compilers, not people. Your job
is to make the code _readable_ and _understandable_. If you need performance,
you can think of a better sequence of steps, not some "let's make this a mess

10
content/books/things-i-learnt/patterns-not-solutions/index.md

@ -14,10 +14,10 @@ the problem it self -- to fit the pattern.
My guess is that the heavy use of "let's apply _this_ design pattern" before
even understanding the problem -- or even trying to solve it -- comes as a
form of [cargo cult](/books/things-i-learnt/cargo-cult): I heard people used
this pattern and solved their problem, so let's use it too and it will solve
our problem. Or, worse: Design pattern is described by _Famous Person_, so we
must use it.
form of [cargo cult](/books/things-i-learnt/cargo-cult): "We saw that people
used this pattern and solved their problem, so let's use it too and it will
solve our problem". Or, worse: "Design pattern is described by _Famous
Person_, so we must use it".
Here is the thing: Design pattern should _not_ be used as a way to find
solution to any problems. You may use some of them as base for your solution,
@ -25,7 +25,7 @@ but you must focus on the _problem_, not the _pattern_.
"Do a visitor pattern will solve this?" is the wrong question. "What should we
do to solve our problem?" is the real question. Once you went there and solved
the problem you may look and see if it is a visitor pattern -- or whatever
the problem you may look back and see if it is a visitor pattern -- or whatever
pattern. If it doesn't, that's alright, 'cause you _solved the problem_. If it
did... well, congratulations, you now know how to name your solution.

5
content/books/things-i-learnt/spec-first/index.md

@ -19,8 +19,9 @@ writing a bunch of things that doesn't solve anything -- or, at least,
anything that _should_ be solved.
So here is the point: Try to get a small spec on whatever you want to solve.
But be aware that even that spec may have to be thrown out, as the
understanding of the problem tend to grow as long as the project continue.
But be aware that even that spec may have to be [thrown
out](/books/things-i-learnt/throw-away), as the understanding of the problem
tend to grow as long as the project continue.
Yes, it's paradoxical: You need a spec to know what to code to avoid coding
the wrong solution, but the spec may be wrong, so you _end up_ solving the

11
content/books/things-i-learnt/steps-as-comments/index.md

@ -15,17 +15,18 @@ There you are, looking at the blank file wondering how you're going to solve
that problem. Here is a tip:
Take the spec you (or someone else) wrote. Break each point into a series of
steps to reach the expected content. You can even write on your natural
steps to reach the expected behaviour. You can even write on your natural
language, if you don't speak English.
Then fill the spaces between the comments with code.
For example, if you have a spec of "connect to server X and retrieve
everything there. You should save the content in the database. Remember that
server X has an API that you can pass an ID (the last ID seen) and you can use
it to not retrieve the same content again." Pretty simple, right?
everything there. Save the content in the database. Remember that server X API
allow you can pass an ID (the last ID seen) and you can use it to not retrieve
the same content again." Pretty simple, right?
Now, writing this in comments, pointing the steps you need to make:
Writing this as comments, pointing the steps you need to make, you may end up
with something like this:
```
// connect to server X

34
content/books/things-i-learnt/tests-dead-code/index.md

@ -14,46 +14,48 @@ keeps changing, how will you know when a function is not necessary anymore?
Writing a test for every single function on your system may come from the
"100% Coverage Syndrome", which afflicts some managers, thinking that the only
way to be completely sure your system is "bug free" is to write tests for
every single piece of code, till you reach the magical "100% coverage" in all
every single line of code, till you reach the magical "100% coverage" in all
the tests.
I do believe you can reach 100% coverage, as long as you're willing to
_delete_ your code.
Cue the universal grasps here.
(Cue the universal grasps here.)
But how do you know which pieces of code can be deleted?
When I mentioned [integration
tests](/books/things-i-learnt/integration-tests), I mentioned how much more
sense it made to me reading them instead of the "unit" tests, because they
were describing exactly how the system would operate in normal conditions. If
you write tests the go through the system, doing normal operations, and you
can get tests for all the normal cases -- and some "abnormal", like when
things go wrong -- then you know that, if you run those tests and they mark
some lines as "not tested", it's because you don't need them.
were describing exactly how the system would operate in normal (and some
abnormal) conditions. If you write tests that go through the system, assuming
it is a black box with an input point and an output, and you can get tests for
all the normal cases -- and some "abnormal", like when things go wrong -- then
you know that, if you run those tests and they mark some lines as "not
tested", it's because you don't need them.
"But Julio, you're forgetting the error control!" I do agree, specially when
you're talking with project owners or some other expert, that people will
forget to tell you what to do in case of things going wrong -- say, someone
entering a name in the age field -- but _you_ can see those and _you_ know
forget to tell you what to do in case of things going wrong -- say, the user
typing their name in the age field -- but _you_ can see those and _you_ know
that you need error control so _you_ can add the error control and describe
the situation where that error control would trigger.
If, on the other hand, you write a test for every function, when you do a
short/simple check, you'll find that the function is still being used in the
system -- by the tests, not actually, "value to the user" code. Sure, you can
system by the tests, not actually, "value to the user" code. Sure, you can
use your IDE to go back and forth between code and test and see if it points a
use beyond the test, but it won't do it for yourself.
There is one other weird thing about trying to write integration tests for
error controls: Sometimes, you can't reach the control. It's true! I did wrote
control checks for every function once but, when running in the integration
tests, there was no way to produce an input at the input layer of the system
that would reach the error control in that function -- mostly 'cause the
There is one other weird thing about using integration tests for error
controls: Sometimes, you can't reach the control statement. It's true! I did
wrote control checks for every function once but, when running in the
integration tests, there was no way to produce an input at the input layer of
the system that would reach the error control in that function 'cause the
other functions, which would run before the one I was trying to test, would
catch the error before it. If that's a design problem or not -- it probably
was -- it's a different discussion, but the fact is that that function didn't
need error control.
need error control, something that I wouldn't see if I wrote test specifically
for it, but it was clear in an integration test run.
{{ chapters(prev_chapter_link="/books/things-i-learnt/integration-tests", prev_chapter_title="Unit Tests Are Good, Integration Tests Are Gooder", next_chapter_title="Tests Make Better APIs", next_chapter_link="/books/things-i-learnt/tests-apis") }}

Loading…
Cancel
Save