jbiason
/
Presentations
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

Merge branch 'release/20191211'

master
Julio Biason 5 years ago
parent
1a20a491ed 3ac12beb74
commit
2c4c388285
2 changed files with 639 additions and 661 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 389
      intro-stream-processing.html
  2. 911
      porque-como-opensource.html

389
intro-stream-processing.html
Unescape View File

@ -76,30 +76,26 @@
</style> </style>
</head> </head>
<body> <body>
<div class="reveal"> <div class="reveal">
<div class="slides"> <div class="slides">
<section> <section data-background="_images/streamprocessing.jpg" data-header>
<section data-background="_images/streamprocessing.jpg" data-header> <h2 class="semi-opaque">Uma Não-Gentil Introdução ao Stream Processing</h2>
<h2 class="semi-opaque">Uma Não-Gentil Introdução ao Stream Processing</h2> </section>
</section>
</section> <section>
<img src="_images/avatar-20170726.png" alt="Me" style="float:left;width:200px;" class="no-border">
<section>
<section> <div>
<img src="_images/avatar-20170726.png" alt="Me" style="float:left;width:200px;" class="no-border"> <ul class="empty">
<li>Júlio Biason</li>
<div> <li><a href="https://functional.cafe/@juliobiason">https://functional.cafe/@juliobiason</a></li>
<ul class="empty"> <li><a href="https://t.me/juliobiason">https://t.me/juliobiason</a></li>
<li>Júlio Biason</li> <li>julio.biason@pm.me</li>
<li><a href="https://functional.cafe/@juliobiason">https://functional.cafe/@juliobiason</a></li> <li><a href="http://presentations.juliobiason.net">http://presentations.juliobiason.net</a></li>
<li><a href="https://t.me/juliobiason">https://t.me/juliobiason</a></li> </ul>
<li>julio.biason@pm.me</li> </div>
<li><a href="http://presentations.juliobiason.net">http://presentations.juliobiason.net</a></li> </section>
</ul>
</div>
</section>
</section>
<section> <section>
<h2>Agenda:</h2> <h2>Agenda:</h2>
@ -115,7 +111,7 @@
<section> <section>
<section> <section>
<h2>This is a CDN</h2> <h2>This is a CDN</h2>
<img class="stretch" src="_images/streamprocessing-aws.png" alt="Lista de pontos de presença da CDN da Amazon" /> <img class="stretch" src="_images/streamprocessing-aws.png" alt="Lista de pontos de presença da CDN da Amazon" />
<aside class="notes"> <aside class="notes">
@ -136,7 +132,7 @@
<section> <section>
<h2>This is a CDN</h2> <h2>This is a CDN</h2>
<p>A AWS tem um limite máximo de 100.000 requisições/segundo.</p> <p>A AWS tem um limite máximo de 100.000 requisições/segundo.</p>
</section> </section>
@ -166,7 +162,7 @@
banco de dados e fazer um SELECT pra ver o total. banco de dados e fazer um SELECT pra ver o total.
</p> </p>
<p class="fragment">Precisamos os dados "read ready".</p> <p class="fragment">Precisamos os dados "<strong>read ready</strong>".</p>
</section> </section>
<section> <section>
@ -201,9 +197,7 @@
<p class="fragment">... a não ser que registros antigos fosse apagados.</p> <p class="fragment">... a não ser que registros antigos fosse apagados.</p>
<p class="fragment">... que é o que serviços de mensageria fazem.</p> <p class="fragment">... que é o que serviços de mensageria fazem.</p>
</section> </section>
</section>
<section>
<section data-background="_images/streamprocessing-message-broker.png"> <section data-background="_images/streamprocessing-message-broker.png">
<h1 class="semi-opaque">Message Brokers</h1> <h1 class="semi-opaque">Message Brokers</h1>
</section> </section>
@ -217,12 +211,12 @@
clientes: clientes:
- Um dos clientes é um "Producer", responsável pela - Um dos clientes é um "Producer", responsável pela
geração de dados; numa analogia com banco de dados, geração de dados; numa analogia com banco de dados,
seria um aplicativo que somente fizesse INSERRTS. seria um aplicativo que somente fizesse INSERRTS.
- O outro cliente é o "Consumer", responsável por - O outro cliente é o "Consumer", responsável por
ler os dados gerados os dados gerados pelo ler os dados gerados os dados gerados pelo
Producer. Producer.
</aside> </aside>
</section> </section>
@ -255,10 +249,10 @@
</section> </section>
<section> <section>
<img src="_images/streamprocessing-kafka.png" alt="Kafka Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-kafka.png" alt="Kafka Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-rabbitmq.png" alt="RabbitMQ Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-rabbitmq.png" alt="RabbitMQ Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-sqs.png" alt="AWS SQS Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-sqs.png" alt="AWS SQS Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-zeromq.gif" alt="ZeroMQ Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-zeromq.gif" alt="ZeroMQ Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<aside class="notes"> <aside class="notes">
Os Mesage Brokers mais usados são: Os Mesage Brokers mais usados são:
@ -281,9 +275,7 @@
Lambda. Lambda.
</aside> </aside>
</section> </section>
</section>
<section>
<section data-background="_images/streamprocessing-monolith.jpg"> <section data-background="_images/streamprocessing-monolith.jpg">
<h1 class="fragment semi-opaque">Batch Processing</h1> <h1 class="fragment semi-opaque">Batch Processing</h1>
</section> </section>
@ -320,21 +312,19 @@
<section> <section>
<img class="stretch plain" src="_images/dunno.jpg" alt="" /> <img class="stretch plain" src="_images/dunno.jpg" alt="" />
<aside class="notes"> <aside class="notes">
Esse é todo o conhecimento que eu tenho de batch Esse é todo o conhecimento que eu tenho de batch
processing, simplesmente porque eu pulei processing, simplesmente porque eu pulei
diretamente para stream processing sem nunca ter diretamente para stream processing sem nunca ter
feito muita coisa com batch processing. feito muita coisa com batch processing.
</aside> </aside>
</section> </section>
<section> <section>
<img src="_images/streamprocessing-spark.png" alt="Apache Spark Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-spark.png" alt="Apache Spark Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-hadoop.png" alt="Apache Hadoop Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-hadoop.png" alt="Apache Hadoop Logo" style="float:left;width:200px;margin:50px;" class="no-border">
</section> </section>
</section>
<section>
<section data-background="_images/streamprocessing-functional.jpg"> <section data-background="_images/streamprocessing-functional.jpg">
<h1 class="semi-opaque">Programação Funcional</h1> <h1 class="semi-opaque">Programação Funcional</h1>
</section> </section>
@ -363,7 +353,7 @@
<section> <section>
<pre><code>def mult(a): <pre><code>def mult(a):
return a * 4</code></pre> return a * 4</code></pre>
</section> </section>
<section> <section>
@ -376,42 +366,42 @@
<section> <section>
<pre><code>class LightSwitch: <pre><code>class LightSwitch:
def __init__(self): def __init__(self):
self.state = False self.state = False
def switch(self): def switch(self):
self.state = not self.state self.state = not self.state
print(self.state)</code></pre> print(self.state)</code></pre>
</section> </section>
<section> <section>
<pre><code>light = LightSwitch() <pre><code>light = LightSwitch()
light.switch()</code></pre> light.switch()</code></pre>
<span class="fragment">True</span> <span class="fragment">True</span>
<pre class="fragment"><code>light.switch()</code></pre> <pre class="fragment"><code>light.switch()</code></pre>
<span class="fragment">False</span> <span class="fragment">False</span>
</section> </section>
<section> <section>
<pre><code>print()</code></pre> <pre><code>print()</code></pre>
<aside class="notes"> <aside class="notes">
Por mais estranho que possa soar, `print()` também Por mais estranho que possa soar, `print()` também
não é uma função pura, porque chamar print() duas não é uma função pura, porque chamar print() duas
vezes vai deixar duas linhas na tela. vezes vai deixar duas linhas na tela.
</aside> </aside>
<pre class="fragment"><code>INSERT INTO table (value, value)</code></pre> <pre class="fragment"><code>INSERT INTO table (value, value)</code></pre>
<aside class="notes"> <aside class="notes">
Insert também não é considerada uma função pura, Insert também não é considerada uma função pura,
porque cada vez que é chamada adiciona um novo porque cada vez que é chamada adiciona um novo
registro na tabela. registro na tabela.
</aside> </aside>
<pre class="fragment"><code>random()</code></pre> <pre class="fragment"><code>random()</code></pre>
</section> </section>
<section> <section>
<p>"Idempotência"</p> <p>"Idempotência"</p>
@ -423,110 +413,110 @@ light.switch()</code></pre>
</aside> </aside>
</section> </section>
<section> <section>
<h2>Imutabilidade</h2> <h2>Imutabilidade</h2>
<p>Uma vez que o dado é gerado, ele nunca muda.</p> <p>Uma vez que o dado é gerado, ele nunca muda.</p>
<p class="fragment"><small>(Transparência referencial)</small></p> <p class="fragment"><small>(Transparência referencial)</small></p>
</section> </section>
<section> <section>
<img src="_images/lisp.png" alt=""> <img src="_images/lisp.png" alt="">
<aside class="notes"> <aside class="notes">
Mas uma coisa legal que seguiu a teoria das Mas uma coisa legal que seguiu a teoria das
categorias, implementando programação funcional, categorias, implementando programação funcional,
foi Lisp. foi Lisp.
</aside> </aside>
</section> </section>
<section> <section>
<ul> <ul>
<li><code>map</code></li> <li><code>map</code></li>
<li><code>reduce</code> <span class="fragment">(fold)</span></li> <li><code>reduce</code> <span class="fragment">(fold)</span></li>
<li class="fragment"><code>filter</code></li> <li class="fragment"><code>filter</code></li>
</ul> </ul>
</section> </section>
<section> <section>
<ul> <ul>
<li><code>(map lambda iterable)</code></li> <li><code>(map lambda iterable)</code></li>
<li><code>(fold lambda iterable start)</code></li> <li><code>(fold lambda iterable start)</code></li>
<li><code>(filter lambda iterable)</code></li> <li><code>(filter lambda iterable)</code></li>
</ul> </ul>
</section> </section>
<section> <section>
<ul> <ul>
<li><code><strong>map</strong></code>: transforma cada um dos <li><code><strong>map</strong></code>: transforma cada um dos
elementos do iterador através do função lambda elementos do iterador através do função lambda
em uma nova lista.</li> em uma nova lista.</li>
<li><code><strong>fold</strong></code>: converte todos os elementos <li><code><strong>fold</strong></code>: converte todos os elementos
da lista em um único valor, começando com um da lista em um único valor, começando com um
valor adicionar (por exemplo, <code>sum</code>).</li> valor adicionar (por exemplo, <code>sum</code>).</li>
<li><code><strong>filter</strong></code>: remove elementos do <li><code><strong>filter</strong></code>: remove elementos do
iterador que não sejam verdadeiros pelo lambda iterador que não sejam verdadeiros pelo lambda
e produz uma nova lista. </li> e produz uma nova lista. </li>
</ul> </ul>
</section> </section>
</section> </section>
<section> <section>
<section data-background="_images/streamprocessing.jpg"> <section data-background="_images/streamprocessing.jpg">
<h1 class="semi-opaque">Stream Processing</h1> <h1 class="semi-opaque">Stream Processing</h1>
</section> </section>
<section> <section>
<pre><code>sudo cat /etc/shadow | cut -d ':' -f 2 | sort | uniq -c</code></pre> <pre><code>sudo cat /etc/shadow | cut -d ':' -f 2 | sort | uniq -c</code></pre>
</section> </section>
<section> <section>
<pre><code>sudo cat [broker] | cut -d ':' -f 2 | sort | uniq -c</code></pre> <pre><code>sudo cat [broker] | cut -d ':' -f 2 | sort | uniq -c</code></pre>
</section> </section>
<section> <section>
<pre><code>sudo cat [broker] | map | sort | uniq -c</code></pre> <pre><code>sudo cat [broker] | map | sort | uniq -c</code></pre>
</section> </section>
<section> <section>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
</section> </section>
<section> <section>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
<pre><code>sudo cat [broker] | map | fold</code></pre> <pre><code>sudo cat [broker] | map | fold</code></pre>
</section> </section>
<section> <section>
<pre><code>sudo cat [broker] | map | group_by | fold</code></pre> <pre><code>sudo cat [broker] | map | group_by | fold</code></pre>
<pre><code>sudo cat [broker] | map | group_by | fold</code></pre> <pre><code>sudo cat [broker] | map | group_by | fold</code></pre>
<pre><code>sudo cat [broker] | map | group_by | fold</code></pre> <pre><code>sudo cat [broker] | map | group_by | fold</code></pre>
<pre><code>sudo cat [broker] | map | group_by | fold</code></pre> <pre><code>sudo cat [broker] | map | group_by | fold</code></pre>
<pre><code>sudo cat [broker] | map | group_by | fold</code></pre> <pre><code>sudo cat [broker] | map | group_by | fold</code></pre>
</section> </section>
<section> <section>
<img class="plain stretch" src="_images/streamprocessing-streamprocessing.png" alt=""/> <img class="plain stretch" src="_images/streamprocessing-streamprocessing.png" alt=""/>
</section> </section>
<section> <section>
<img class="plain stretch" src="_images/streamprocessing-pipeline.png" alt="" /> <img class="plain stretch" src="_images/streamprocessing-pipeline.png" alt="" />
</section> </section>
<section> <section>
<img class="plain stretch" src="_images/unclephil-flinkpipelinereal.png" alt="" /> <img class="plain stretch" src="_images/unclephil-flinkpipelinereal.png" alt="" />
</section> </section>
<section> <section>
<img src="_images/streamprocessing-storm.png" alt="Apache Storm Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-storm.png" alt="Apache Storm Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-flink.png" alt="Apache Flink Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-flink.png" alt="Apache Flink Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-wallaroo.png" alt="Wallaroo Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-wallaroo.png" alt="Wallaroo Logo" style="float:left;width:200px;margin:50px;" class="no-border">
<img src="_images/streamprocessing-beam.png" alt="Apache Beam Logo" style="float:left;width:200px;margin:50px;" class="no-border"> <img src="_images/streamprocessing-beam.png" alt="Apache Beam Logo" style="float:left;width:200px;margin:50px;" class="no-border">
</section> </section>
</section> </section>
<section> <section>
@ -572,9 +562,9 @@ light.switch()</code></pre>
</section> </section>
<section> <section>
"A distributed system is one in which the failure of a "A distributed system is one in which the failure of a
computer you didn't even know existed can render your computer you didn't even know existed can render your
own computer unusable" own computer unusable"
-- Leslie Lamport, 1987 -- Leslie Lamport, 1987
</section> </section>
@ -606,52 +596,51 @@ light.switch()</code></pre>
graças ao jupyter. graças ao jupyter.
</aside> </aside>
</section> </section>
</section>
<section data-background='_images/thats-all-folks.jpg'>
<ul class="empty fragment semi-opaque">
<li><a href="https://functional.cafe/@juliobiason">https://functional.cafe/@juliobiason</a></li>
<li><a href="https://t.me/juliobiason">https://t.me/juliobiason</a></li>
<li>julio.biason@pm.me</li>
<li><a href="http://presentations.juliobiason.net">http://presentations.juliobiason.net</a></li>
</ul>
<h3></h3>
<h3 class="semi-opaque fragment">
Perguntas?
</h3>
</section>
</div>
</div>
<script src="reveal.js/lib/js/head.min.js"></script> <section data-background='_images/thats-all-folks.jpg'>
<script src="reveal.js/js/reveal.js"></script> <ul class="empty fragment semi-opaque">
<li><a href="https://functional.cafe/@juliobiason">https://functional.cafe/@juliobiason</a></li>
<script> <li><a href="https://t.me/juliobiason">https://t.me/juliobiason</a></li>
// Full list of configuration options available at: <li>julio.biason@pm.me</li>
// https://github.com/hakimel/reveal.js#configuration <li><a href="http://presentations.juliobiason.net">http://presentations.juliobiason.net</a></li>
Reveal.initialize({ </ul>
controls: true,
progress: true,
history: true,
center: true,
// showNotes: true,
transition: 'fade', // none/fade/slide/convex/concave/zoom
// Optional reveal.js plugins
dependencies: [
{ src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'reveal.js/plugin/zoom-js/zoom.js', async: true },
{ src: 'reveal.js/plugin/notes/notes.js', async: true }
]
});
</script>
</body> <h3></h3>
<h3 class="semi-opaque fragment">
Perguntas?
</h3>
</section>
</div>
</div>
<script src="reveal.js/lib/js/head.min.js"></script>
<script src="reveal.js/js/reveal.js"></script>
<script>
// Full list of configuration options available at:
// https://github.com/hakimel/reveal.js#configuration
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
// showNotes: true,
transition: 'fade', // none/fade/slide/convex/concave/zoom
// Optional reveal.js plugins
dependencies: [
{ src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'reveal.js/plugin/zoom-js/zoom.js', async: true },
{ src: 'reveal.js/plugin/notes/notes.js', async: true }
]
});
</script>
</body>
</html> </html>

911
porque-como-opensource.html
View File

File diff suppressed because it is too large Load Diff
Write Preview
Loading…
Cancel
Save