Skip to content

Planet scale, cloud-edge native big data engine crafted for heterogeneous workloads

License

Notifications You must be signed in to change notification settings

jiangxinmeng1/matrixone

 
 

Repository files navigation

Connect with us:

matrixone16 matrixone16

If you are interested in MatrixOne project, please kindly give MatrixOne a triple `Star`, `Fork` and `Watch`, Thanks!

Contents

What is MatrixOne?

MatrixOne is a hyper-converged cloud & edge native distributed database with a structure that separates storage, computation, and transactions to form a consolidated HSTAP data engine. This engine enables a single database system to accommodate diverse business loads such as OLTP, OLAP, and stream computing. It also supports deployment and utilization across public, private, and edge clouds, ensuring compatibility with diverse infrastructures.

MatrixOne

💥 Hyper-converged Engine

Monolithic Engine HTAP data engine that supports a mix of workloads such as TP, AP, time series, and machine learning within a single database.
Built-in Streaming Engine Built-in stream computing engine that enables real-time data inflow, transformation, and querying.

☁️ Cloud & Edge Native

Storage-Computation Separation Structure Separates the storage, computation, and transaction layers, leveraging a containerized design for ultimate scalability.
Multi-Infrastructure Compatibility MatrixOne provides industry-leading latency control with optimized consistency protocol.

🚀 Extreme Performance

High-Performance Execution Engine The flexible combination of Compute Node and Transaction node accommodates point queries and batch processing, delivering peak performance for OLTP and OLAP.
Enterprise-Grade High Availability Establishes a consistently shared log under a leading Multi-Raft replication state machine model. It ensures high cluster availability while preventing data duplication, thus achieving RTO=0.

🖊️ Ease of Use

Built-in Multi-Tenancy Capability Offers inherent multi-tenancy, where tenants are isolated from each other, independently scalable yet uniformly manageable. This feature simplifies the complexity of multi-tenancy design in upper-level applications.
High Compatibility with MySQL MatrixOne exhibits high compatibility with MySQL 8.0, including transmission protocol, SQL syntax, and ecosystem tools, lowering usage and migration barriers.

💰 Cost-Effective

Efficient Storage Design Employs cost-effective object storage as primary storage. High availability can be achieved through erasure coding technology with only about 150% data redundancy. It also provides high-speed caching capabilities, balancing cost and performance via a multi-tiered storage strategy that separates hot and cold data.
Flexible Resource Allocation Users can adjust the resource allocation ratio for OLTP and OLAP according to business conditions, maximizing resource utilization.

🔒 Enterprise-Level Security and Compliance

MatrixOne employs Role-Based Access Control (RBAC), TLS connections, and data encryption to establish a multi-tiered security defense system, safeguarding enterprise-level data security and compliance.

Simplify Database Management and Maintenance With business evolution, the number of data engines and middleware enterprises employ increases. Each data engine relies on 5+ essential components and stores 3+ data replicas. Each engine must be independently installed, monitored, patched, and upgraded. This results in high and uncontrollable data engine selection, development, and operations costs. Under MatrixOne's unified architecture, users can employ a single database to serve multiple data applications, reducing the number of introduced data components and technology stacks by 80% and significantly simplifying database management and maintenance costs.
Reduce Data Fragmentation and Inconsistency Data flow and copy between databases make data sync and consistency increasingly tricky. The unified and incrementally materialized view of MatrixOne allows the downstream to support real-time upstream updates and achieve end-to-end data processing without redundant ETL processes.
Decoupling Data Architecture From Infrastructure Currently, the architecture design across different infrastructures is complicated, causing new data silos between cloud and edge, cloud and on-premise. MatrixOne is designed with a unified architecture to support simplified data management and operations across different infrastructures.
Extremely Fast Complex Query Performance Poor business agility results from slow, complex queries and redundant intermediate tables in current data warehousing solutions. MatrixOne supports blazing-fast experience even for star and snowflake schema queries, improving business agility with real-time analytics.
Solid OLTP-like OLAP Experience Current data warehousing solutions have the following problems: high latency and absence of immediate visibility for data updates. MatrixOne brings OLTP (Online Transactional Processing) level consistency and high availability to CRUD operations in OLAP (Online Analytical Processing).
Seamless and Non-disruptive Scalability It is challenging to balance performance and scalability to achieve an optimum price-performance ratio in current data warehousing solutions. MatrixOne's disaggregated storage and compute architecture makes it fully automated and efficient to scale in/out and up/down without disrupting applications.

MatrixOne's architecture is as below:

MatrixOne

For more details, you can checkout MatrixOne Architecture Design.

⚙️ Install MatrixOne

MatrixOne supports Linux and MacOS. You can install MatrixOne either by building from source or using docker. For other installation types, please refer to MatrixOne installation for more details.

Step 1.Install Dependency

  • Building from source
  1. Install Go (version 1.22 is required)

    Click Go Download and install to enter its official documentation, and follow the installation steps to complete the Go installation.

  2. Install GCC/Clang

    Click GCC Download and install to enter its official documentation, and follow the installation steps to complete the GCC installation.

  3. Install Git

    Install Git via the Official Documentation.

  4. Install and configure MySQL Client

    Click MySQL Community Downloads to enter into the MySQL client download and installation page. According to your operating system and hardware environment.Configure the MySQL client environment variables.

  • Using docker
  1. Download and install Docker

Click Get Docker, enter into the Docker's official document page, depending on your operating system, download and install the corresponding Docker. It is recommended to choose Docker version 20.10.18 or later and strive to maintain consistency between the Docker client and Docker server versions.

  1. Install and configure MySQL Client

Click MySQL Community Downloads to enter into the MySQL client download and installation page. According to your operating system and hardware environment.Configure the MySQL client environment variables.

Note: MySQL client version 8.0.30 or later is recommended.

Step 2. Install the mo_ctl tool

mo_ctl is a command-line tool for deploying, installing, and managing MatrixOne. It is very convenient to perform various operations on MatrixOne.

The mo_ctl tool can be installed through the following command:

wget https://raw.githubusercontent.com/matrixorigin/mo_ctl_standalone/main/install.sh && sudo -u $(whoami) bash +x ./install.sh

See mo_ctl Tool for complete usage details.

Step 3. Set mo_ctl parameters

  • Building from source
mo_ctl set_conf MO_PATH="yourpath" # Set custom MatrixOne download path
mo_ctl set_conf MO_DEPLOY_MODE=git # Set MatrixOne deployment method
  • Using docker
mo_ctl set_conf MO_CONTAINER_DATA_HOST_PATH="/yourpath/mo/" # Set the data directory for host
mo_ctl set_conf MO_DEPLOY_MODE=docker # Set MatrixOne deployment method

Step 4. One-click install Matrixone

Depending on your needs, choose whether you want to keep your code up to date, or if you want to get the latest stable version of the code.

  • Option 1: Get the MatrixOne(Develop Version)

The main branch is the default branch, the code on the main branch is always up-to-date but not stable enough.

mo_ctl deploy main
  • Option 2: Get the MatrixOne(Stable Version)

If you want to get the latest stable version code released by MatrixOne, Please specify a specific version number.

mo_ctl deploy <latest version>

Step 5. Launch MatrixOne server

Launch the MatrixOne service through the mo_ctl start command.

Tips: The initial startup of MatrixOne approximately takes 20 to 30 seconds. After a brief wait, you can connect to MatrixOne using the MySQL client.

Step 6. Connect to MatrixOne

One-click connection to MatrixOne service through mo_ctl connect command.

Note: The login account in the above code snippet is the initial account; please change the initial password after logging in to MatrixOne; see Password Management.

Contributions to MatrixOne are welcome from everyone.
See Contribution Guide for details on submitting patches and the contribution workflow.

👏 All contributors

nnsgmsone
Nnsgmsone
XuPeng-SH
XuPeng-SH
zhangxu19830126
Fagongzi
reusee
Reusee
ouyuanning
Ouyuanning
daviszhen
Daviszhen
aunjgr
BRong Njam
sukki37
Maomao
iamlinjunhong
Iamlinjunhong
jiangxinmeng1
Jiangxinmeng1
jianwan0214
Jianwan0214
LeftHandCold
GreatRiver
w-zr
Wei Ziran
m-schen
Chenmingsong
dengn
Dengn
aptend
Aptend
lni
Lni
xzxiong
Jackson
YANGGMM
YANGGMM
qingxinhome
Qingxinhome
badboynt1
Nitao
broccoliSpicy
BroccoliSpicy
mooleetzi
Mooleetzi
fengttt
Fengttt
zzl200012
Kutori
lacrimosaprinz
Prinz
guguducken
Brown
dongdongyang33
Dongdongyang
JackTan25
Boyu Tan
cnutshell
Cui Guoke
JinHai-CN
Jin Hai
lignay
Matthew
bbbearxyz
Bbbearxyz
tianyahui-python
Tianyahui-python
wanglei4687
Wanglei
triump2020
Triump2020
heni02
Heni02
wanhanbo
Wanhanbo
iceTTTT
IceTTTT
volgariver6
LiuBo
taofengliu
刘陶峰
Ariznawlll
Ariznawlll
goodMan-code
GoodMan-code
yingfeng
Yingfeng
mklzl
Mklzl
jensenojs
Jensen
domingozhang
DomingoZhang
arjunsk
Arjun Sunil Kumar
chrisxu333
Nuo Xu
aressu1985
Aressu1985
matrix-meow
Mo-bot
zengyan1
Zengyan1
aylei
Aylei
noneback
NoneBack
WenhaoKong2001
Otter
richelleguice
Richelle Guice
yjw1268
Ryan
e1ijah1
Elijah
MatrixAdventurer
MatrixAdventurer
NTH19
NTH19
anitajjx
Anitajjx
whileskies
Whileskies
BePPPower
BePPPower
jiajunhuang
Jiajun Huang
Morranto
Morranto
Y7n05h
Y7n05h
songjiayang
Songjiayang
Abirdcfly
Abirdcfly
decster
Binglin Chang
Charlie17Li
Charlie17Li
DanielZhangQD
DanielZhangQD
Juneezee
Eng Zer Jun
ericsyh
Eric Shen
Fungx
Fungx
player-kirito
Kirito
JasonPeng1310
Jason Peng
ikenchina
O2
RinChanNOWWW
RinChanNOW!
TheR1sing3un
TheR1sing3un
chaixuqing
XuQing Chai
qqIsAProgrammer
Yiliang Qiu
yubindy
ZeYu Zhao
adlternative
ZheNing Hu
TszKitLo40
Zijie Lu
ZoranPandovski
Zoran Pandovski
yegetables
Ajian
bxiiiiii
Binxxi
coderzc
Coderzc
forsaken628
ColdWater
dr-lab
Dr-lab
florashi181
Florashi181
hiyoyolumi
Hiyoyolumi
jinfuchiang
Jinfu
sourcelliu
Liuguangliang
lokax
Lokax
lyfer233
Lyfer233
sundy-li
Sundyli
supermario1990
Supermario1990
lawrshen
Tjie
Toms1999
Toms
wuliuqii
Wuliuqii
xiw5
Xiyuedong
yclchuxue
Yclchuxue
ZtXavier
Zt

MatrixOne is licensed under the Apache License, Version 2.0.

About

Planet scale, cloud-edge native big data engine crafted for heterogeneous workloads

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 96.3%
  • PLpgSQL 1.9%
  • Yacc 1.2%
  • C 0.2%
  • Python 0.1%
  • Assembly 0.1%
  • Other 0.2%