NextViper Data Subsystem API Reference

Module: `data`

The data module provides utilities for creating, loading, inspecting, transforming, and processing tabular data and numerical arrays.

`data.load(path: string) -> DataFrame`

Loads a dataset from disk, automatically detecting format by file extension (.csv, .json, .jsonl).

import data

let df = data.load("users.csv")

`data.read_csv(content_or_path: string) -> DataFrame`

Parses CSV content from a string or reads a CSV file from a path.

let df = data.read_csv("name,age
Alice,30
Bob,25
")

`data.read_json(content_or_path: string) -> DataFrame`

Parses JSON array of objects or line-delimited JSON (JSONL).

let df = data.read_json("[{"name": "Alice", "age": 30}]")

`data.array(values: list[float|int]) -> DataArray`

Creates a numerical array from a list of numbers.

let arr = data.array([1.0, 2.0, 3.0, 4.0, 5.0])

`data.zeros(shape: list[int]|int) -> DataArray`

Creates an array of zeros with the specified shape.

let z = data.zeros([100, 4])

`data.ones(shape: list[int]|int) -> DataArray`

Creates an array of ones with the specified shape.

let o = data.ones([10, 10])

`data.arange(start: float, stop: float, step: float = 1.0) -> DataArray`

Generates evenly spaced values within a given half-open interval [start, stop).

let r = data.arange(0.0, 10.0, 0.5)

`data.linspace(start: float, stop: float, num: int = 50) -> DataArray`

Generates num evenly spaced numbers over the closed interval [start, stop].

let grid = data.linspace(0.0, 1.0, 100)

NextViper Data Subsystem Tutorial: From CSV to Model-Ready Data

In this tutorial, you will learn how to load a dataset, inspect its structure, clean missing values, transform numeric features, and prepare train/test splits for training.

Create a sample CSV file housing.csv:

rooms,sqft,age,price
3,1200.0,10,250000.0
4,1850.0,5,380000.0
2,850.0,null,175000.0
5,2400.0,2,490000.0
3,1350.0,15,270000.0

Load the CSV into NextViper:

import data
import std.io

// 1. Load the dataset
let df = data.load("housing.csv")

// 2. Inspect dimensions and columns
io.print("Shape (rows, cols):", df.shape)
io.print("Columns:", df.columns)

// 3. View the first 3 rows
let preview = df.head(3)
io.print(preview.to_csv())

Output:

Shape (rows, cols): [5, 4]
Columns: ["rooms", "sqft", "age", "price"]
rooms,sqft,age,price
3,1200,10,250000
4,1850,5,380000
2,850,,175000

Real-world datasets often contain missing (null) entries. You can clean them with .drop_missing():

// Remove all rows with missing values
let clean_df = df.drop_missing()
io.print("Clean rows count:", clean_df.num_rows)

Output:

Clean rows count: 4

Generate statistical descriptions (count, mean, min, max, std):

let stats = clean_df.describe()
io.print("Mean square footage:", stats["sqft"]["mean"])
io.print("Max price:", stats["price"]["max"])

Before training machine learning models, features should be normalized and split into training and test partitions:

// 1. Min-Max normalize numeric columns to [0.0, 1.0]
let normalized_df = clean_df.normalize()

// 2. Deterministically shuffle data with seed
let shuffled_df = normalized_df.shuffle(42)

// 3. Split into 80% train and 20% test sets
let split_pair = shuffled_df.split(0.8, 42)
let train_set = split_pair[0]
let test_set = split_pair[1]

io.print("Training set rows:", train_set.num_rows)
io.print("Testing set rows:", test_set.num_rows)

Convert preprocessed features into Tensor instances ready for neural networks:

let feature_cols = ["rooms", "sqft", "age"]
let train_tensors = train_set.to_tensor(feature_cols)

io.print("Tensor Shape:", train_tensors.shape)

Output:

Tensor Shape: [3, 3]

You are now ready to feed your preprocessed data directly into the NextViper AI and neural network pipelines!