kwarray.util_random module

Handle and interchange between different random number generators (numpy, python, torch, …). Also defines useful random iterator functions and ensure_rng().

Random Number Generator Patterns

If you need a seeded random number generator kwarray.ensure_rng is helpful with that: kwarray.util_random.ensure_rng()

If the input is a number it returns a seeded random number generator. If it is None is returns whatever the system level RNG is. If the input is an existing RNG it returns it without changing it. It also has the ability to switch between Python’s random module RNG and numpys np.random RNG (it can translate the internal state between the two).

When I write randomized functions / class, a coding pattern I like is to have a default keyword argument rng=None. Then kwarray.ensure_rng coerces whatever the input is into a random.Random() or numpy.random.RandomState() object.

def some_random_function(*args, rng=None):
    rng = kwarray.ensure_rng(rng)

Then if this random function calls any other random function, it passes the coerced rng to all other subfunctions. This ensures that seeing the RNG at the top level produces a completely determenistic process.

For a more involved example

import pandas as pd
import numpy
import kwarray

def random_subfunc1(rng=None):
    rng = kwarray.ensure_rng(rng, api='python')
    value: float = rng.betavariate(3, 2.3)
    return value


def random_subfunc2(rng=None):
    rng = kwarray.ensure_rng(rng, api='numpy')
    arr: np.ndarray = rng.choice([1, 2, 3, 4], size=3, replace=0)
    return arr


def random_method(rng=None):
    value = random_subfunc1(rng=rng)
    arr = random_subfunc2(rng=rng)
    final = (arr * value).sum()
    return final

def demo():
    results = []
    num = 10
    for _ in range(num):
        rng = np.random.RandomState(3)
        row = {}
        row['system'] = random_method(None)
        row['seeded'] = random_method(0)
        row['exiting'] = random_method(rng)
        results.append(row)

    df = pd.DataFrame(results)
    print(df)

This results in:

     system    seeded   exiting
0  3.642700  6.902354  4.869275
1  3.127890  6.902354  4.869275
2  4.317397  6.902354  4.869275
3  3.382259  6.902354  4.869275
4  1.999498  6.902354  4.869275
5  5.293688  6.902354  4.869275
6  2.984741  6.902354  4.869275
7  6.455160  6.902354  4.869275
8  5.161900  6.902354  4.869275
9  2.810358  6.902354  4.869275
kwarray.util_random.seed_global(seed, offset=0)[source]

Seeds the python, numpy, and torch global random states

Parameters:
  • seed (int) – seed to use

  • offset (int) – if specified, uses a different seed for each global random state separated by this offset. Defaults to 0.

kwarray.util_random.shuffle(items, rng=None)[source]

Shuffles a list inplace and then returns it for convinience

Parameters:
  • items (list | ndarray) – data to shuffle

  • rng (int | float | None | numpy.random.RandomState | random.Random) – seed or random number gen

Returns:

this is the input, but returned for convinience

Return type:

list

Example

>>> list1 = [1, 2, 3, 4, 5, 6]
>>> list2 = shuffle(list(list1), rng=1)
>>> assert list1 != list2
>>> result = str(list2)
>>> print(result)
[3, 2, 5, 1, 4, 6]
kwarray.util_random.random_combinations(items, size, num=None, rng=None)[source]

Yields num combinations of length size from items in random order

Parameters:
  • items (List) – pool of items to choose from

  • size (int) – Number of items in each combination

  • num (int | None) – Number of combinations to generate. If None, generate them all.

  • rng (int | float | None | numpy.random.RandomState | random.Random) – seed or random number generator. Defaults to the global state of the python random module.

Yields:

Tuple – a random combination of items of length size.

Example

>>> # xdoctest: +REQUIRES(module:scipy)
>>> import ubelt as ub
>>> items = list(range(10))
>>> size = 3
>>> num = 5
>>> rng = 0
>>> # xdoctest: +IGNORE_WANT
>>> combos = list(random_combinations(items, size, num, rng))
>>> print('combos = {}'.format(ub.urepr(combos, nl=1)))
combos = [
    (0, 6, 9),
    (4, 7, 8),
    (4, 6, 7),
    (2, 3, 5),
    (1, 2, 4),
]

Example

>>> # xdoctest: +REQUIRES(module:scipy)
>>> import ubelt as ub
>>> items = list(zip(range(10), range(10)))
>>> # xdoctest: +IGNORE_WANT
>>> combos = list(random_combinations(items, 3, num=5, rng=0))
>>> print('combos = {}'.format(ub.urepr(combos, nl=1)))
combos = [
    ((0, 0), (6, 6), (9, 9)),
    ((4, 4), (7, 7), (8, 8)),
    ((4, 4), (6, 6), (7, 7)),
    ((2, 2), (3, 3), (5, 5)),
    ((1, 1), (2, 2), (4, 4)),
]
kwarray.util_random.random_product(items, num=None, rng=None)[source]

Yields num items from the cartesian product of items in a random order.

Parameters:
  • items (List[Sequence]) – items to get caresian product of packed in a list or tuple. (note this deviates from api of itertools.product())

  • num (int | None) – maximum number of items to generate. If None generat them all

  • rng (int | float | None | numpy.random.RandomState | random.Random) – Seed or random number generator. Defaults to the global state of the python random module.

Yields:

Tuple – a random item in the cartesian product

Example

>>> import ubelt as ub
>>> items = [(1, 2, 3), (4, 5, 6, 7)]
>>> rng = 0
>>> # xdoctest: +IGNORE_WANT
>>> products = list(random_product(items, rng=0))
>>> print(ub.urepr(products, nl=0))
[(3, 4), (1, 7), (3, 6), (2, 7),... (1, 6), (2, 5), (2, 4)]
>>> products = list(random_product(items, num=3, rng=0))
>>> print(ub.urepr(products, nl=0))
[(3, 4), (1, 7), (3, 6)]

Example

>>> # xdoctest: +REQUIRES(--profile)
>>> rng = ensure_rng(0)
>>> items = [np.array([15, 14]), np.array([27, 26]),
>>>          np.array([21, 22]), np.array([32, 31])]
>>> num = 2
>>> for _ in range(100):
>>>     list(random_product(items, num=num, rng=rng))
kwarray.util_random._npstate_to_pystate(npstate)[source]

Convert state of a NumPy RandomState object to a state that can be used by Python’s Random. Derived from [SO44313620].

References

Example

>>> py_rng = random.Random(0)
>>> np_rng = np.random.RandomState(seed=0)
>>> npstate = np_rng.get_state()
>>> pystate = _npstate_to_pystate(npstate)
>>> py_rng.setstate(pystate)
>>> assert np_rng.rand() == py_rng.random()
kwarray.util_random._pystate_to_npstate(pystate)[source]

Convert state of a Python Random object to state usable by NumPy RandomState. Derived from [SO44313620].

References

Example

>>> py_rng = random.Random(0)
>>> np_rng = np.random.RandomState(seed=0)
>>> pystate = py_rng.getstate()
>>> npstate = _pystate_to_npstate(pystate)
>>> np_rng.set_state(npstate)
>>> assert np_rng.rand() == py_rng.random()
kwarray.util_random._coerce_rng_type(rng)[source]

Internal method that transforms input seeds into an integer form.

kwarray.util_random.ensure_rng(rng=None, api='numpy')[source]

Coerces input into a random number generator.

This function is useful for ensuring that your code uses a controlled internal random state that is independent of other modules.

If the input is None, then a global random state is returned.

If the input is a numeric value, then that is used as a seed to construct a random state.

If the input is a random number generator, then another random number generator with the same state is returned. Depending on the api, this random state is either return as-is, or used to construct an equivalent random state with the requested api.

Parameters:
  • rng (int | float | None | numpy.random.RandomState | random.Random) – if None, then defaults to the global rng. Otherwise this can be an integer or a RandomState class. Defaults to the global random.

  • api (str) – specify the type of random number generator to use. This can either be ‘numpy’ for a numpy.random.RandomState object or ‘python’ for a random.Random object. Defaults to numpy.

Returns:

rng - either a numpy or python random number generator, depending on the setting of api.

Return type:

(numpy.random.RandomState | random.Random)

Example

>>> rng = ensure_rng(None)
>>> ensure_rng(0).randint(0, 1000)
684
>>> ensure_rng(np.random.RandomState(1)).randint(0, 1000)
37

Example

>>> num = 4
>>> print('--- Python as PYTHON ---')
>>> py_rng = random.Random(0)
>>> pp_nums = [py_rng.random() for _ in range(num)]
>>> print(pp_nums)
>>> print('--- Numpy as PYTHON ---')
>>> np_rng = ensure_rng(random.Random(0), api='numpy')
>>> np_nums = [np_rng.rand() for _ in range(num)]
>>> print(np_nums)
>>> print('--- Numpy as NUMPY---')
>>> np_rng = np.random.RandomState(seed=0)
>>> nn_nums = [np_rng.rand() for _ in range(num)]
>>> print(nn_nums)
>>> print('--- Python as NUMPY---')
>>> py_rng = ensure_rng(np.random.RandomState(seed=0), api='python')
>>> pn_nums = [py_rng.random() for _ in range(num)]
>>> print(pn_nums)
>>> assert np_nums == pp_nums
>>> assert pn_nums == nn_nums

Example

>>> # Test that random modules can be coerced
>>> import random
>>> import numpy as np
>>> ensure_rng(random, api='python')
>>> ensure_rng(random, api='numpy')
>>> ensure_rng(np.random, api='python')
>>> ensure_rng(np.random, api='numpy')