main

2021/04/02

different data pipelines in data science projects

 There are many combinations which software stack you can choose for your data science projects. Here I am listing a few of them. Then I will create some examples to show how to glue them together.


1) redis, flower, celery, flask


2) elasticsearch/kibana, Jupiter for development, logstash, minio, nifi, rabbitmq, grafana, redis, faas, prometheus, alertmanager, flask


3) ES/Kibana, minio, nifi, rabbitmq, Casandra, hive, redis, MISP, keycloak, MongoDB,flask


4) LDAP, portioner, gâteau, drone, rabbitmq, kibana, minio, traefik



-

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

How to Supercharge Your Python Classes with Class Methods

  How to Supercharge Your Python Classes with Class Methods | by Siavash Yasini | May, 2024 | Towards Data Science As we just mentioned, a c...

  • A Guide to Python's Magic Methods
     https://rszalski.github.io/magicmethods/ __init__ __repr__ __getattr__ __setattr__ __call__(self, [args...]) context manager class Closer: ...
  • Asynchronous Tasks Using Flask, Redis, and Celery
    Use  Celery Celery is an asynchronous task queue based on distributed message passing to distribute workload across machines or threads. A c...
  • hands on with minio
    https://docs.min.io/docs/python-client-quickstart-guide.html The MinIO Python Client SDK provides simple APIs to access any Amazon S3 com...
  • super charged neurons in AI
    I am working in the Artificial Intelligence area. I am thinking of two events happened to me. 1) I have the same dream over many years. Wh...
  • Mac OS Log Analysis (1)
    Recently, my Mac laptop's (Majove, MacBook Pro (15-inch, 2017)) Safari failed to open. I searched around and could not find a solution. ...