API Reference for pystackql

StackQL Class

class pystackql.StackQL(server_mode=False, server_address='127.0.0.1', server_port=5466, output='dict', sep=',', header=False, debug=False, debug_log_file=None, **kwargs)[source]

Bases: object

A class representing an instance of the StackQL query engine.

Parameters:

  • server_mode (bool, optional) – Connect to a StackQL server (defaults to False)

  • server_address (str, optional) – The address of the StackQL server (server_mode only, defaults to ‘127.0.0.1’)

  • server_port (int, optional) – The port of the StackQL server (server_mode only, defaults to 5466)

  • backend_storage_mode (str, optional) – Specifies backend storage mode, options are ‘memory’ and ‘file’ (defaults to ‘memory’, this option is ignored in server_mode)

  • backend_file_storage_location (str, optional) – Specifies location for database file, only applicable when backend_storage_mode is ‘file’ (defaults to ‘{cwd}/stackql.db’, this option is ignored in server_mode)

  • output (str, optional) – Determines the format of the output, options are ‘dict’, ‘pandas’, ‘csv’, and ‘markdownkv’ (defaults to ‘dict’, ‘csv’ is not supported in server_mode, ‘markdownkv’ is optimized for LLM understanding)

  • sep (str, optional) – Seperator for values in CSV output (defaults to ‘,’, output=’csv’ only)

  • header (bool, optional) – Show column headers in CSV output (defaults to False, output=’csv’ only)

  • download_dir (str, optional) – The download directory for the StackQL executable (defaults to site.getuserbase(), not supported in server_mode)

  • app_root (str, optional) – Application config and cache root path (defaults to {cwd}/.stackql)

  • execution_concurrency_limit (int, optional) – Concurrency limit for query execution (defaults to -1 - unlimited)

  • dataflow_dependency_max (int, optional) – Max dataflow weakly connected components for a given query (defaults to 50)

  • dataflow_components_max (int, optional) – Max dataflow components for a given query (defaults to 50)

  • api_timeout (int, optional) – API timeout (defaults to 45, not supported in server_mode)

  • proxy_host (str, optional) – HTTP proxy host (not supported in server_mode)

  • proxy_password (str, optional) – HTTP proxy password (only applicable when proxy_host is set)

  • proxy_port (int, optional) – HTTP proxy port (defaults to -1, only applicable when proxy_host is set)

  • proxy_scheme (str, optional) – HTTP proxy scheme (defaults to ‘http’, only applicable when proxy_host is set)

  • proxy_user (str, optional) – HTTP proxy user (only applicable when proxy_host is set)

  • max_results (int, optional) – Max results per HTTP request (defaults to -1 for no limit, not supported in server_mode)

  • page_limit (int, optional) – Max pages of results that will be returned per resource (defaults to 20, not supported in server_mode)

  • max_depth (int, optional) – Max depth for indirect queries: views and subqueries (defaults to 5, not supported in server_mode)

  • custom_registry (str, optional) – Custom StackQL provider registry URL (e.g. https://registry-dev.stackql.app/providers) supplied using the class constructor

  • custom_auth (dict, optional) – Custom StackQL provider authentication object supplied using the class constructor (not supported in server_mode)

  • debug (bool, optional) – Enable debug logging (defaults to False)

  • debug_log_file (str, optional) – Path to debug log file (defaults to ~/.pystackql/debug.log, only available if debug is True)

— Read-Only Attributes —

Parameters:

  • platform (str, readonly) – The operating system platform

  • package_version (str, readonly) – The version number of the pystackql Python package

  • version (str, readonly) – The version number of the stackql executable (not supported in server_mode)

  • params (list, readonly) – A list of command-line parameters passed to the stackql executable (not supported in server_mode)

  • bin_path (str, readonly) – The full path of the stackql executable (not supported in server_mode).

  • sha (str, readonly) – The commit (short) sha for the installed stackql binary build (not supported in server_mode).

__init__(server_mode=False, server_address='127.0.0.1', server_port=5466, output='dict', sep=',', header=False, debug=False, debug_log_file=None, **kwargs)[source]

Constructor method

properties()[source]

Retrieves the properties of the StackQL instance.

This method collects all the attributes of the StackQL instance and returns them in a dictionary format.

Returns:

A dictionary containing the properties of the StackQL instance.

Return type:

dict

Example

{
    "platform": "Darwin x86_64 (macOS-12.0.1-x86_64-i386-64bit), Python 3.10.9",
    "output": "dict",
    ...
}
upgrade(showprogress=True)[source]

Upgrades the StackQL binary to the latest version available.

This method initiates an upgrade of the StackQL binary. Post-upgrade, it updates the version and sha attributes of the StackQL instance to reflect the newly installed version.

Parameters:

showprogress (bool, optional) – Indicates if progress should be displayed during the upgrade. Defaults to True.

Returns:

A message indicating the new version of StackQL post-upgrade.

Return type:

str

executeStmt(query, custom_auth=None, env_vars=None, **kwargs)[source]
Executes a query using the StackQL instance and returns the output as a string.

This is intended for operations which do not return a result set, for example a mutation operation such as an INSERT or a DELETE or life cycle method such as an EXEC operation or a REGISTRY PULL operation.

This method determines the mode of operation (server_mode or local execution) based on the server_mode attribute of the instance. If server_mode is True, it runs the query against the server. Otherwise, it executes the query using a subprocess.

Parameters:

  • query (str, list of dict objects, or Pandas DataFrame) – The StackQL query string to be executed.

  • custom_auth (dict, optional) – Custom authentication dictionary.

  • env_vars (dict, optional) – Command-specific environment variables for this execution.

  • kwargs (optional) – Additional keyword arguments that override constructor parameters for this execution. Supported overrides: output, sep, header, auth, custom_registry, max_results, page_limit, max_depth, api_timeout, http_debug, proxy_host, proxy_port, proxy_user, proxy_password, proxy_scheme, backend_storage_mode, backend_file_storage_location, app_root, execution_concurrency_limit, dataflow_dependency_max, dataflow_components_max

Returns:

The output result of the query in string format. If in server_mode, it returns a JSON string representation of the result.

Return type:

dict, Pandas DataFrame or str (for csv output)

Example

>>> from pystackql import StackQL
>>> stackql = StackQL()
>>> stackql_query = "REGISTRY PULL okta"
>>> result = stackql.executeStmt(stackql_query)
>>> result
execute(query, suppress_errors=True, custom_auth=None, env_vars=None, **kwargs)[source]

Executes a StackQL query and returns the output based on the specified output format.

This method supports execution both in server mode and locally using a subprocess. In server mode, the query is sent to a StackQL server, while in local mode, it runs the query using a local binary.

Parameters:

  • query (str) – The StackQL query string to be executed.

  • suppress_errors (bool, optional) – If set to True, the method will return an empty list if an error occurs.

  • custom_auth (dict, optional) – Custom authentication dictionary.

  • env_vars (dict, optional) – Command-specific environment variables for this execution.

  • kwargs (optional) – Additional keyword arguments that override constructor parameters for this execution. Supported overrides: output, sep, header, auth, custom_registry, max_results, page_limit, max_depth, api_timeout, http_debug, proxy_host, proxy_port, proxy_user, proxy_password, proxy_scheme, backend_storage_mode, backend_file_storage_location, app_root, execution_concurrency_limit, dataflow_dependency_max, dataflow_components_max

Returns:

The output of the query, which can be a list of dictionary objects, a Pandas DataFrame, or a raw CSV string, depending on the configured output format.

Return type:

list(dict) | pd.DataFrame | str

Raises:

ValueError – If an unsupported output format is specified.

Example:
>>> stackql = StackQL()
>>> query = '''
... SELECT SPLIT_PART(machineType, '/', -1) as machine_type, status, COUNT(*) as num_instances
... FROM google.compute.instances
... WHERE project = 'stackql-demo' AND zone = 'australia-southeast1-a'
... GROUP BY machine_type, status HAVING COUNT(*) > 2
... '''
>>> result = stackql.execute(query)
async executeQueriesAsync(queries)[source]

Executes multiple StackQL queries asynchronously using the current StackQL instance.

This method utilizes an asyncio event loop to concurrently run a list of provided StackQL queries. Each query is executed independently, and the combined results of all the queries are returned as a list of JSON objects if ‘dict’ output mode is selected, or as a concatenated DataFrame if ‘pandas’ output mode is selected.

The order of the results in the returned list or DataFrame may not necessarily correspond to the order of the queries in the input list due to the asynchronous nature of execution.

Parameters:

queries (list[str], required) – A list of StackQL query strings to be executed concurrently.

Returns:

A list of results corresponding to each query. Each result is a JSON object or a DataFrame.

Return type:

list[dict] or pd.DataFrame

Raises:

  • ValueError – If server_mode is True (async is only supported in local mode).

  • ValueError – If an unsupported output mode is selected (anything other than ‘dict’ or ‘pandas’).

Example

>>> from pystackql import StackQL
>>> stackql = StackQL()
>>> queries = [
>>> """SELECT '%s' as region, instance_type, COUNT(*) as num_instances
... FROM aws.ec2.instances
... WHERE region = '%s'
... GROUP BY instance_type""" % (region, region)
>>> for region in regions ]
>>> result = stackql.executeQueriesAsync(queries)

Note

  • This method is only supported in local mode.

test_connection()[source]

Tests if the server connection is working by executing a simple query.

This method is only valid when server_mode=True.

Returns:

True if the connection is working, False otherwise.

Return type:

bool

Raises:

ValueError – If called when not in server mode.