mlpath.mlquest package

Submodules

mlpath.mlquest.mlquest module

This is the main module of mlquest. It contains the mlquest class which is used to log machine learning experiments.

class mlpath.mlquest.mlquest.mlquest

Bases: object

The mlquest class provides methods and attributes to log machine learning experiments.

static start_quest(quest_name, **kwargs)

Start a new run under the quest with quest_name. This function should be called before any other function with logging functionality.

Parameters

quest_name – The name of the experiment this run belongs to (e.g, the name of the model in the current file)

Example

The following would start a new quest called ‘Naive-Bayes’

>>> start_quest('Naive-Bayes')

static clear()

Clear the log record of the current run. You may use it while handling exceptions or debugging.

static l(func, name=None)

Log the scalar parameters of a function. This function should be called on any function that you want to log the parameters of.

Parameters

• func (function) – The function to be logged

• name (string) – A custom name of the function to be logged. If not given, the name of the function will be used.

Returns

The function wrapped with the logging functionality

Example

The following would log the parameters of the function NaiveBayesFit in the current run log

>>> accuracy = mlq.l(NaiveBayesFit)(alpha=1024, beta_param=7, c=12, )

Notes

• It doesn’t matter whether the argument is given through a variable or as a value, it doesn’t matter if its given as a named argument or not. mlq.l() will log the values under the column corresponding to the name as in the function’s signature.

• mlq.l() always tracks all scalar arguments given to a function that have a name using the function’s signature

• If you later make a new function then MLQuest may handle this by creating new columns that are empty for the previous runs (rows).

• Likewise, deleting a function will make the corresponding columns empty for the future runs (rows).

• mlq.l() doesn’t log collections to avoid having to deal with very large arrays. If your hyperparameter is a small array then you can still stringify it and log it using the mlq.to_log_ext() method

static log_metrics(m1=None, m2=None, m3=None, m4=None, m5=None, m6=None, m7=None, m8=None, m9=None, m10=None, **kwargs)

Log the metrics of the experiment. As an experimental feature, if the metrics are given as positional arguments, they will be logged with the name of the variable given to them. If they are given as keyword arguments, they will be logged with the name as the keyword.

Parameters

mi (scalar) – The ith metric to be logged

Example

>>> acc = mlq.l(NaiveBayes)(alpha=1024, beta_param=7, c=12, )
>>> mlq.log_metrics(acc)

This would log the accuracy of the NaiveBayes under the column ‘acc’. To provide a different name than that of the variable you can use the keyword argument syntax:

>>> mlq.log_metrics(accuracy=acc)

Notes

• Your metric should be a scalar. You may need to convert a Numpy array into a scalar by using the metric.item().

• You can log multiple metrics at once using this function

static to_log(col_name, dict=None, **kwargs)

Grants logging with extensive access to the log.

Parameters

• col_name (string) – The name of the column to log to

• dict – A dictionary of the key (subcolumns), values to be logged under col_name column

• kwargs – key value pairs to be llogged under col_name column (an alternative to dict)

Example

>>> mlq.to_log('graphs', Scatterplot='../plots/plt21.jpg', Histogram='../plots/plt22.jpg')

This would log the Scatterplot and Histogram under the ‘graphs’ column. Any previous runs will have empty values for these columns.

static end_quest(save_ext=None, blacklist=[], log_defs=False)

ends an active run and internally saves it to the log. This must called at the end of the experiment else it will not be logged.

Parameters

• (optional) (log_defs) – Where to save the log externally. Defaults to not saving externally at all (None).

• (optional) – A list of columns to not log. Defaults to an empty list. The column can be passed as ‘col_name’ or ‘header.col_name’ if there is a clash in names.

• (optional) – If true, it adds all the default columns that are not explictly passed to the blacklist.

Example

static show_logs(*args, last_k=None, highlight='yellow', **kwargs)

Shows the logs of a quest in a table that can be rendered in a jupyter notebook.

Parameters

• (optional) (highlight) – The number of (most recent) experiments to show. Defaults to all experiments.

• (optional) – The color to highlight the most recent experiment with. Defaults to yellow.

Example

>>> mlq.show_logs('NaiveBayesExp')

static run_server()

Runs the server to display the logs of the quests in a web browser. This includes all the quests in the current directory.

static delete_runs(run_ids)

permanently deletes runs that have ids in run_ids from the log.

Parameters

run_ids (list of ints) – The ids (indecies) of the runs to be deleted

Example

>>> mlq.delete_runs([1, 2, 3])

This would delete the runs with ids 1, 2, and 3 from the current quest

static get_flat_dict(show_all=False)

Convert the quests table to a flat dictionary. This is helpful if the table is needed in a csv or dataframe format.

Parameters

show_all – If True, the dictionary will contain all the columns in the table. If False, it will obey the blacklist and log_defs sent to end_quest.

static delete_quest(quest_name)

mlpath.mlquest.utils module

Utility functions for mlquest

Module contents