DataHighland Studio
A browser workspace with a SQL editor, a catalog of your tables, jobs with run history and logs, and VS Code or JupyterLab on your projects.
Open source · Built on AWS
Own your data lake.
Your own data lakehouse, in your own AWS account, at AWS prices. Notebooks, SQL, a catalog of your tables and scheduled pipelines, with nothing running when nobody is working.
A lakehouse you own, at the price of the cloud
Managed lakehouse platforms are great, and they charge a premium on top of the cloud. If your product runs on a lot of data and every dollar counts, that premium is the problem.
DataHighland gives you the same way of working, built from plain AWS services in your own account. You pay AWS directly and nothing on top. Your data stays in your bucket, as open Iceberg tables any engine can read.
What you get
Notebooks with Spark ready
Python, PySpark and SQL cells. Spark is there the moment a notebook opens, signed in as you.
Pipelines as code
PySpark jobs, dbt models and an Airflow DAG per project, in git. Schedules live in the code; CI deploys them.
Open tables
Apache Iceberg on S3 with the AWS Glue catalog. Any engine that reads Iceberg can read your data.
Near-zero idle cost
No always-on cluster, no NAT gateway. Machines start for work and stop by themselves.
Private by default
Production data readable only inside your network, by known roles. Only CI deploys; only jobs write; ingested data is signed.
How it works
Your AWS account
- LandingFiles your sources push: JSON, CSV, Parquet
- JobsSpark, dbt and Airflow on a machine that starts on schedule and stops when done
- Iceberg tablesIn your S3 bucket, listed in the Glue catalog
- DataHighland StudioSQL, notebooks, catalog and jobs for you and your team
Each data project is its own git repository. Push to main, and CI publishes it and sets its schedules. At the scheduled time a small machine starts, runs what's due, records the run and stops.
What it costs
$0DataHighland fee. You pay AWS directly for what you use.
Idle = offMachines start for work and stop by themselves. Storage is most
of an idle bill.
Your callHow often your jobs run decides most of the bill, and you choose
it.
Measured monthly costs, for a mostly idle setup and a busy one, come with the first release.
Early, and built in public
Everything above works today, in one AWS account set up by hand. The first milestone is a one-command install into yours.
- One-command install into your own AWS account
- Getting-started guide and a sample project
- Measured monthly costs, idle and busy
- Bigger machines on demand for heavy jobs
- Failure alerts by email
Who's behind it
DataHighland is created by Fahad, a data engineer who built it for his own data products, and is maintained by Greenvale IT Solutions.
Want DataHighland set up in your AWS account, your pipelines built, or the whole thing run for you? Get in touch.