Spark Connect for Rust

A Spark Connect client implementation in Rust. This repository provides both a Rust library for interacting with Spark Connect APIs and also a simple command line for sending Spark SQL commands to any Spark Connect endpoint.

Table of Contents

• Building
• Using
  • CLI
  • In Rust

Building

Building can be done with cargo by invoking cargo build and cargo test

Using

CLI

The spark-connect-cli binary can be used to run Spark SQL commands against the Spark Connect server, for example:

% cargo run --bin spark-connect-cli
>> create temporary view people as select current_date();
+-------------+
| show_string |
+-------------+
| (0 rows)    |
|             |
+-------------+
>> select * from people;
+-------------------------------+
| show_string                   |
+-------------------------------+
| -RECORD 0-------------------- |
|  current_date() | 2023-07-06  |
|                               |
+-------------------------------+
>>

In Rust

Consult the Rustdocs for tips on how to use the library.