Esses dias eu estava fazendo um workshop sobre conceitos de Google Cloud Computing para um cliente, e quando chegamos no BigQuery (DW do GCP + outras coisas), uma das perguntas foi: “eu tenho uma tabela em que algumas colunas não podem ser visíveis para todos. Tenho como limitar isso para alguns grupos sem ter que me preocupar em criar views, ou outra tabela?”

Image for post
Image for post

Minha resposta foi: “já li alguma coisa a respeito, mas volto depois com a resposta mais concreta.”. Fui pesquisar e cai nisso aqui:

https://cloud.google.com/bigquery/docs/column-level-security-intro

Retornei para o cliente com essa documentação, mas falei: “vou fazer uns testes…


Hey you!!! Happy new year!! 2020 was a strange year, but I can’t complain. Lot of things, knowledge, worked a lot. So, let’s keep going in 2021!!!

Image for post
Image for post

So, let’s talk a little about security. I’ve been working within an interesting project to make GCP Security Command Center to work with some external tools and automation with Cloud Functions. And guess what: as I love automation, here’s what I’ve being studying:


Hey you!!! Happy new year!! 2020 was a strange year, but I can’t complain. Lot of things, knowledge, worked a lot. So, let’s keep going in 2021!!!

Image for post
Image for post

So, let’s talk a little about security. I’ve been working within an interesting project to make GCP Security Command Center to work with some external tools and automation with Cloud Functions. And guess what: as I love automation, here’s what I’ve being studying:


I’ve finally got some “free” time to learn a little more about Machine Learning. So, that said, I’m almost crazy with Math (again, same feeling of 90's).

Image for post
Image for post

So, I'm using this to study:

It has been really productive, even for me (I'm not a developer, so it's hard sometimes to go ahead with a few topics). So, my starting point was to run the Titanic study case from Kaggle but using now Bigquery…


Image for post
Image for post

Data Pipeline is basically move some data from one place to another. This move may have or not changes, processing, …

For companies that works with big ammount of data, this is basic job. Data is received, transformed, enriched with other data if needed, moved to data lakes or any other place, and most of times, finished with some beautiful dashboard.

We have in Google Cloud the Data Fusion service. Basically is a service for efficiently building ETL/ELT data pipelines. It uses Cloud Dataproc cluster to perform all transforms in the pipeline. …


If you don’ know the best way to use it, of course!

Image for post
Image for post

BigQuery it’s the Data Warehouse solution fom Google Cloud. And it’s REALLY amazing what you can do with this tool!!! But at same time, it can take you to ruin!! Some of great points about BigQuery is that is serverless, highly-scalable and you can plugin a lot of sources to analyze.

There’s also a lot of main points like multi-cloud analytics, machine learning with SQL, GIS, BI, Sheets connections, … sky is the limit to analysis.

The only point here is about the pricing. BigQuery charges for data…


Terraform is great. It helps you keep things clear and allow you to provision almost everything in the Cloud (I know, not only in Cloud, but this is what I've being using).

Image for post
Image for post
Photo by Sai Kiran Anagani on Unsplash

One more challenge that I took to myself when I got "in charge" for Terraform in a new project… 1 month to certificate (less, actually). As I did about AWS on this post, again, I plan my "journey" to take the HashiCorp Certified: Terraform Associate Certification.


Image for post
Image for post

What kind of name is Kinesis? Well, looks like it means "movement in response of an external stimulus". And in fact, it makes sense… in AWS, Kinesis works as Data Streams, so, it will "receive" something and "supply" this something.


Image for post
Image for post

Ansible is a really great tool. Used to keep your configuration management aligned between your hosts. I really love the idempotence term.

In old days, we used to work like this: go to server, copy and paste what we want, restart service, cross the fingers, then check if ok, logout, and go to next server. Repeat 100 times.

So we just evolve and start to script and iterate through some list of hosts.

A few problems on this is that each one like to script in some language. Some in Shell, some in Python, … and read other…

Marcelo Marques

enjoy the journey. do the right thing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store