API Reference for pystackql

StackQL Class

class pystackql.StackQL(server_mode=False, server_address='127.0.0.1', server_port=5466, backend_storage_mode='memory', backend_file_storage_location='stackql.db', download_dir=None, app_root=None, execution_concurrency_limit=1, output='dict', custom_registry=None, custom_auth=None, sep=',', header=False, api_timeout=45, proxy_host=None, proxy_password=None, proxy_port=-1, proxy_scheme='http', proxy_user=None, max_results=-1, page_limit=20, max_depth=5, debug=False, debug_log_file=None)[source]

Bases: object

A class representing an instance of the StackQL query engine.

Parameters:

server_mode (bool, optional) – Connect to a StackQL server (defaults to False)
server_address (str, optional) – The address of the StackQL server (server_mode only, defaults to ‘127.0.0.1’)
server_port (int, optional) – The port of the StackQL server (server_mode only, defaults to 5466)
backend_storage_mode (str, optional) – Specifies backend storage mode, options are ‘memory’ and ‘file’ (defaults to ‘memory’, this option is ignored in server_mode)
backend_file_storage_location (str, optional) – Specifies location for database file, only applicable when backend_storage_mode is ‘file’ (defaults to ‘{cwd}/stackql.db’, this option is ignored in server_mode)
output (str, optional) – Determines the format of the output, options are ‘dict’, ‘pandas’, and ‘csv’ (defaults to ‘dict’, ‘csv’ is not supported in server_mode)
sep (str, optional) – Seperator for values in CSV output (defaults to ‘,’, output=’csv’ only)
header (bool, optional) – Show column headers in CSV output (defaults to False, output=’csv’ only)
download_dir (str, optional) – The download directory for the StackQL executable (defaults to site.getuserbase(), not supported in server_mode)
app_root (str, optional) – Application config and cache root path (defaults to {cwd}/.stackql)
execution_concurrency_limit (int, optional) – Concurrency limit for query execution (defaults to 1, set to -1 for unlimited)
api_timeout (int, optional) – API timeout (defaults to 45, not supported in server_mode)
proxy_host (str, optional) – HTTP proxy host (not supported in server_mode)
proxy_password (str, optional) – HTTP proxy password (only applicable when proxy_host is set)
proxy_port (int, optional) – HTTP proxy port (defaults to -1, only applicable when proxy_host is set)
proxy_scheme (str, optional) – HTTP proxy scheme (defaults to ‘http’, only applicable when proxy_host is set)
proxy_user (str, optional) – HTTP proxy user (only applicable when proxy_host is set)
max_results (int, optional) – Max results per HTTP request (defaults to -1 for no limit, not supported in server_mode)
page_limit (int, optional) – Max pages of results that will be returned per resource (defaults to 20, not supported in server_mode)
max_depth (int, optional) – Max depth for indirect queries: views and subqueries (defaults to 5, not supported in server_mode)
custom_registry (str, optional) – Custom StackQL provider registry URL (e.g. https://registry-dev.stackql.app/providers) supplied using the class constructor
custom_auth (dict, optional) – Custom StackQL provider authentication object supplied using the class constructor (not supported in server_mode)
debug (bool, optional) – Enable debug logging (defaults to False)
debug_log_file (str, optional) – Path to debug log file (defaults to ~/.pystackql/debug.log, only available if debug is True)

— Read-Only Attributes —

Parameters:

platform (str, readonly) – The operating system platform
package_version (str, readonly) – The version number of the pystackql Python package
version (str, readonly) – The version number of the stackql executable (not supported in server_mode)
params (list, readonly) – A list of command-line parameters passed to the stackql executable (not supported in server_mode)
bin_path (str, readonly) – The full path of the stackql executable (not supported in server_mode).
sha (str, readonly) – The commit (short) sha for the installed stackql binary build (not supported in server_mode).

execute(query, suppress_errors=True)[source]

Executes a StackQL query and returns the output based on the specified output format.

This method supports execution both in server mode and locally using subprocess. In server mode, the query is sent to a StackQL server, while in local mode, it runs the query using a local binary.

Args:

query (str): The StackQL query string to be executed. suppress_errors (bool, optional): If set to True, the method will return an empty list if an error occurs.

Returns:

list(dict), pd.DataFrame, or str: The output of the query, which can be a list of dictionary objects, a Pandas DataFrame, or a raw CSV string, depending on the configured output format.

Raises:

ValueError: If an unsupported output format is specified.

Examples:

>>> stackql = StackQL()
>>> query = '''
        SELECT SPLIT_PART(machineType, '/', -1) as machine_type, status, COUNT(*) as num_instances
        FROM google.compute.instances
        WHERE project = 'stackql-demo' AND zone = 'australia-southeast1-a'
        GROUP BY machine_type, status HAVING COUNT(*) > 2
        '''
>>> result = stackql.execute(query)

async executeQueriesAsync(queries)[source]

Executes multiple StackQL queries asynchronously using the current StackQL instance.

This method utilizes an asyncio event loop to concurrently run a list of provided StackQL queries. Each query is executed independently, and the combined results of all the queries are returned as a list of JSON objects if ‘dict’ output mode is selected, or as a concatenated DataFrame if ‘pandas’ output mode is selected.

The order of the results in the returned list or DataFrame may not necessarily correspond to the order of the queries in the input list due to the asynchronous nature of execution.

Parameters:

queries (list[str], required) – A list of StackQL query strings to be executed concurrently.

Returns:

A list of results corresponding to each query. Each result is a JSON object or a DataFrame.

Return type:

list[dict] or pd.DataFrame

Raises:

ValueError – If method is used in server_mode on an unsupported OS (anything other than Linux).
ValueError – If an unsupported output mode is selected (anything other than ‘dict’ or ‘pandas’).

Example:

>>> from pystackql import StackQL
>>> stackql = StackQL()         
>>> queries = [
>>> """SELECT '%s' as region, instanceType, COUNT(*) as num_instances 
... FROM aws.ec2.instances 
... WHERE region = '%s' 
... GROUP BY instanceType""" % (region, region)
>>> for region in regions ]
>>> result = stackql.executeQueriesAsync(queries)

Note:

When operating in server_mode, this method is not supported.

executeStmt(query)[source]

Executes a query using the StackQL instance and returns the output as a string.: This is intended for operations which do not return a result set, for example a mutation operation such as an INSERT or a DELETE or life cycle method such as an EXEC operation or a REGISTRY PULL operation.

This method determines the mode of operation (server_mode or local execution) based on the server_mode attribute of the instance. If server_mode is True, it runs the query against the server. Otherwise, it executes the query using a subprocess.

Parameters:: query (str, list of dict objects, or Pandas DataFrame) – The StackQL query string to be executed.
Returns:: The output result of the query in string format. If in server_mode, it returns a JSON string representation of the result.
Return type:: dict, Pandas DataFrame or str (for csv output)

Example:

>>> from pystackql import StackQL
>>> stackql = StackQL()
>>> stackql_query = "REGISTRY PULL okta"
>>> result = stackql.executeStmt(stackql_query)
>>> result

properties()[source]

Retrieves the properties of the StackQL instance.

This method collects all the attributes of the StackQL instance and returns them in a dictionary format.

Returns:: A dictionary containing the properties of the StackQL instance.
Return type:: dict

Example:

{
        "platform": "Darwin x86_64 (macOS-12.0.1-x86_64-i386-64bit), Python 3.10.9",
        "parse_json": True,
        ...
}

upgrade(showprogress=True)[source]

Upgrades the StackQL binary to the latest version available.

This method initiates an upgrade of the StackQL binary. Post-upgrade, it updates the version and sha attributes of the StackQL instance to reflect the newly installed version.

Parameters:: showprogress (bool, optional) – Indicates if progress should be displayed during the upgrade. Defaults to True.
Returns:: A message indicating the new version of StackQL post-upgrade.
Return type:: str