Databricks 사용 후기 및 리뷰

A fantastic all-in-one-place platform for big data analytics

주석: We are moving all our big data computation on the Databricks platform. One of the main advantages is the possibility to plan notebooks execution through the workflow section.

장점:

Databricks can be integrated with the most used cloud services (Amazon, Google, Microsoft) and you can create and manages computation clusters, write code and see results in a single platform instead of having multiple separate services. Its dashboard is really easy to use and allows you to focus on code development. It also supports notebooks versioning with GitLab.

단점:

The data visualization part could really be improved.

Databricks: Unleashing Data/AI Potential for Next-Level Analytics and Collaboration

주석: Very good , able to created end-end lakehouse platform for leading healthcare client

장점:

unified lakehouse architecture platform with good collaborationmanaged infrastructure with autoscaling & serverless functionalityautoloader, schema evolution & delta live tablesunit catalog & delta sharing for unified governanceMLflow & integrations with partner connectVersion control databricks repos, secret with security & complianceReal time processing

단점:

Cost & capacity planning for clusters if you don't use DLT/serverless pipelinesPerformance variabilitySome vendor lock-in if you use "DELTA" format with delta live table

Very good to handle very big data

주석: While it supports python, when I need to use it, I ultimately prefer to sample or aggregate and export data to work in another environment. For this end, it works very well.

장점:

- Enables simultaneous collaborative work with colleagues - Easy to mix spark queries and python for extra analyses and plots - Handful visualization modes for query results (tables and plots with aggregations)

단점:

- Hard to manage notebook workspace - Sometimes it gets really slow to run queries - AFAIK, there aren't visualization options for datasets (without running queries)

Best Big Data Analysis Platform

장점:

Databricks is a great platform for working with a huge amount of data. But the most interesting feature is the ability to use Magic query. Magic query allows users to write code in multiple languages in the same notebook.

단점:

Databricks is a go-to platform for most of the analysis and processing workload, but when it comes to the financials, it becomes a little more expensive. And as a result, not a lot of the projects are reliable to be developed in Databricks.

Powerful tool for dev ops of machine learning models

주석: Overall, my experience with Databricks has been very positive. It is a powerful tool to enable data scientists without a lot of data engineering skills. However, you need to be a data scientist or machine learning engineer to be able to take advantage of its power for machine learning.

장점:

I love how easy it is to deploy auto-scaling machine learning models. After a machine learning model is trained, you can just click a button to deploy the model, I believe in a container, and have it auto scale as needed. You can also specify the minimum and maximum size of the deployment to reduce costs but to keep up with the workload as necessary. It is also built around Spark, so tasks involving "big data" aren't an issue.

단점:

Some of the cons are that the primary language is Java/Scala, whereas many data scientists are using python or R, which run slower on Databricks than Java and Scala. Also, the main interface via coding, which can limit a lot of citizen data scientists.

Retrieve SQL query data faster using Databricks.

주석: Databricks to build datalake with ease !

장점:

Easy to use application to build data lakes to run reports on top of it. The Databricks internal architecture helps to run the reports faster.

단점:

The data bricks features are completely wrapper and delivered as snowflake (product) . The Databricks should come up with more features to stand out in the market.

An unified platform to develop high quality analysis

주석: Databricks is allowing data analysis that other systems could not perform at the same performance because it is a platform that integrates huge amounts of cloud data with Scala, Python, SQL or R notebooks in a user-friendly interface. Due to the features of Databricks, daily work seems more efficient and less bureaucratic.

장점:

What I like most about Databricks is the amount of integrations the platform provides to the user. With Databricks, you can create datasets, develop machine learning models, and analyze performance automatically by setting up a job periodically. Whether the user is an engineer, data scientist, or business analyst, Databricks can streamline everyone's work.

단점:

What I least like about Databricks is the instability that usually occurs when there are too many users trying to run their notebooks on the same cluster at the same time.

Modern Analytics with High Flexibility

주석: Positive - after implementing at our company, significant data automation has reduce the amount of time it takes to get in the proper format in the correct people's hands. No more late information that has negative consequences,

장점:

Databricks was able to pull data from our core and create specialized dashboarding / reporting that automated a host of manual process that took hours per week. It is now totally hands off and management get review the data in just a few clicks.

단점:

It can be extremely confusing given the sheer breadth of tools available. The initial setup and connections certainly require an experienced professional, but once up and running, less-technical users can utilize.

Best solution for Enterprise analytics

주석: I've got used to Databricks and writing PySpark codes very easily, people who are not very fluent in coding still can get valuable insights from the data using notebooks developed by data scientists. The ability of databricks to transform data and do whatever you want is powerful.

장점:

The infrastructure is very simple, I have started using Community edition, and then switched to the paid version, however community edition covers most of your needs if you are a student or doing one time projects.

단점:

There are no many users of Pyspark, sometimes finding some answers is hard /there are no many forums, resources, no many questions in Stack overflow/

Databricks Review

주석: Very good. It made analyzing big data a lot easier

장점:

This product has democratized big data computation. Its very easy to move from any platform to this product as it supports most of the languages.

단점:

Nothing so far- may be cost of computation can improve over time but still an economical product to build in-house big data capability.

Prático e intuitivo

장점:

A construção de querys e dashboards são elaboradas de forma prática e intuitiva, ferramenta indispensável para o controle e análise de dados para uma empresa de médio e grande porte.

단점:

Esporadicamente a ferramenta trava, porém não tive muitos problemas em relação a isso recentemente.

Review on databricks

주석: I would strongly recommend this software for others to use their project needs

장점:

I’m one of active user using this software day to day needs its pioneer data store layer by holding transactional process stream line it and hold the information by applying business rules .

단점:

It’s pioneer to to hold the source raw traditions as a refined layer to store the data for longer time

Excellent for data analysis

주석: Excellent. Very fast and easy to use. Also it is easy to get help in the documentation. No lags, and support a big number of users.

장점:

The access and manipulation of data. The software is very fast and great to manipulate and treat data. Also it is possible to build models.

단점:

The lack of options of visualization and creation of dashboards. The creation of dashboards is possible, but is not intuitive.

Great platform for sharing repository

장점:

Our team can collaborate on a project simultaneously and make changes to the scripts. It is fast and reliable.

단점:

Sometimes we need to restart the cluster when system gets crashed.

Powerful tool for data analysis

주석: Overall databricks is very good, and if optimized correctly, can let you work with big scale datasets.

장점:

I love the z indexing, which allows for really fast querying of data. Optimized by spark it is great.

단점:

The data visualization are subpar. I wish there were better libraries to integrate and visual the data.

Great tool for the toolbox

장점:

I'm a SQL person, so being able to run big data analytics in my preferred language was quite nice. Being able to (near) seamlessly swap between Scala, SQL, and python in the same script is quite powerful. If you don't know how to do something easily in one language, do it in another and then swap back. It's pretty performant and querying non-indexed data dumped from the source systems, even if those datasets aren;t quite "big data". I found it to be quicker to dump 100mil rows of staged date from our on-prem server to the data lake and crunch it in Databricks than it was to run in SQL.

단점:

I wasn't involved in the pricing piece, but from what I understand it's fairly expensive. The clusters can be spun up or down as needed, and there's a nice inactivity shutdown feature if you forget to turn off a test cluster, or something. I also had a pretty rough time getting an Azure Gen 2 Data Lake connected, but after finding the not-so-well-documented bug, it wasn't a big deal.

Great solution for your ETL workloads!

주석: We are using Databricks for blockchain ETL workloads

장점:

- Documentation is GREAT - Implementation is mostly straight-forward - The service is easy to use and full of features - Support is top-notch - The interviews we had with the DB guys were more like a peers meeting than a corporate call (I love this) - Our DS lead engineer totally loves it

단점:

- Can't really speak of anything that we don't like about the product at the moment

Great tool to unlock potential from data science teams

주석: Overall I find Databricks to be fantastic tool that I almost couldn't live without. Highly recommend it.

장점:

Databricks allows data science teams to do things that they normally would not be able to do without a much greater level of technical ability. Their mission is "making big data simple" and they definitely deliver on that promise.

단점:

One area where there's still potential to improve further is around making machine learning more accessible. Currently ML still requires a pretty significant degree of data engineering knowledge, but I would love to see Databricks make ML even more accessible.

Good portal for data science related work

장점:

I like the portal page, which connects all Azure subscriptions.

단점:

It can be difficult to understand, and not much tutorial is available.

Databricks Review

주석: Databricks was chosen as part of a new cloud based data platform. Engagement from the company could be better, however the product itself does the job

장점:

Easy to use user interface Can be widely shared across an enterprise with various teams Apache Spark Cluster part of product

단점:

Information Security considerations have to be taken into account due to need for integrations with databricks VPCs when hosted in AWS

All data in one place

장점:

-Open source -Built upon excellent technologies -Broad set of data ingestion sources -Reliable and scalable -Cost efficient data processing

단점:

-Can get overwhelming when you start using it -Would be nice to be able visualize data on the fly

Good python environment

주석: Mainly, i use databricks to run large queries, otherwise, I export the data

장점:

In databricks is easy to transfer the result of a spark query to the python environment, and it has several plots with automatic aggregations

단점:

Databricks has a bad file management system and it is slow sometimes. In addition there are no ways to make a visual query, without using code.