kwarray.util_random module¶
Handle and interchange between different random number generators (numpy,
python, torch, …). Also defines useful random iterator functions and
ensure_rng()
.
Random Number Generator Patterns¶
If you need a seeded random number generator kwarray.ensure_rng is helpful with
that: kwarray.util_random.ensure_rng()
If the input is a number it returns a seeded random number generator. If it is None is returns whatever the system level RNG is. If the input is an existing RNG it returns it without changing it. It also has the ability to switch between Python’s random module RNG and numpys np.random RNG (it can translate the internal state between the two).
When I write randomized functions / class, a coding pattern I like is to
have a default keyword argument rng=None
. Then kwarray.ensure_rng coerces
whatever the input is into a random.Random()
or
numpy.random.RandomState()
object.
def some_random_function(*args, rng=None):
rng = kwarray.ensure_rng(rng)
Then if this random function calls any other random function, it passes the coerced rng to all other subfunctions. This ensures that seeing the RNG at the top level produces a completely determenistic process.
For a more involved example
import pandas as pd
import numpy
import kwarray
def random_subfunc1(rng=None):
rng = kwarray.ensure_rng(rng, api='python')
value: float = rng.betavariate(3, 2.3)
return value
def random_subfunc2(rng=None):
rng = kwarray.ensure_rng(rng, api='numpy')
arr: np.ndarray = rng.choice([1, 2, 3, 4], size=3, replace=0)
return arr
def random_method(rng=None):
value = random_subfunc1(rng=rng)
arr = random_subfunc2(rng=rng)
final = (arr * value).sum()
return final
def demo():
results = []
num = 10
for _ in range(num):
rng = np.random.RandomState(3)
row = {}
row['system'] = random_method(None)
row['seeded'] = random_method(0)
row['exiting'] = random_method(rng)
results.append(row)
df = pd.DataFrame(results)
print(df)
This results in:
system seeded exiting
0 3.642700 6.902354 4.869275
1 3.127890 6.902354 4.869275
2 4.317397 6.902354 4.869275
3 3.382259 6.902354 4.869275
4 1.999498 6.902354 4.869275
5 5.293688 6.902354 4.869275
6 2.984741 6.902354 4.869275
7 6.455160 6.902354 4.869275
8 5.161900 6.902354 4.869275
9 2.810358 6.902354 4.869275
- kwarray.util_random.seed_global(seed, offset=0)[source]¶
Seeds the python, numpy, and torch global random states
- Parameters:
seed (int) – seed to use
offset (int) – if specified, uses a different seed for each global random state separated by this offset. Defaults to 0.
- kwarray.util_random.shuffle(items, rng=None)[source]¶
Shuffles a list inplace and then returns it for convinience
- Parameters:
items (list | ndarray) – data to shuffle
rng (int | float | None | numpy.random.RandomState | random.Random) – seed or random number gen
- Returns:
this is the input, but returned for convinience
- Return type:
Example
>>> list1 = [1, 2, 3, 4, 5, 6] >>> list2 = shuffle(list(list1), rng=1) >>> assert list1 != list2 >>> result = str(list2) >>> print(result) [3, 2, 5, 1, 4, 6]
- kwarray.util_random.random_combinations(items, size, num=None, rng=None)[source]¶
Yields
num
combinations of lengthsize
from items in random order- Parameters:
items (List) – pool of items to choose from
size (int) – Number of items in each combination
num (int | None) – Number of combinations to generate. If None, generate them all.
rng (int | float | None | numpy.random.RandomState | random.Random) – seed or random number generator. Defaults to the global state of the python random module.
- Yields:
Tuple – a random combination of
items
of lengthsize
.
Example
>>> # xdoctest: +REQUIRES(module:scipy) >>> import ubelt as ub >>> items = list(range(10)) >>> size = 3 >>> num = 5 >>> rng = 0 >>> # xdoctest: +IGNORE_WANT >>> combos = list(random_combinations(items, size, num, rng)) >>> print('combos = {}'.format(ub.urepr(combos, nl=1))) combos = [ (0, 6, 9), (4, 7, 8), (4, 6, 7), (2, 3, 5), (1, 2, 4), ]
Example
>>> # xdoctest: +REQUIRES(module:scipy) >>> import ubelt as ub >>> items = list(zip(range(10), range(10))) >>> # xdoctest: +IGNORE_WANT >>> combos = list(random_combinations(items, 3, num=5, rng=0)) >>> print('combos = {}'.format(ub.urepr(combos, nl=1))) combos = [ ((0, 0), (6, 6), (9, 9)), ((4, 4), (7, 7), (8, 8)), ((4, 4), (6, 6), (7, 7)), ((2, 2), (3, 3), (5, 5)), ((1, 1), (2, 2), (4, 4)), ]
- kwarray.util_random.random_product(items, num=None, rng=None)[source]¶
Yields
num
items from the cartesian product of items in a random order.- Parameters:
items (List[Sequence]) – items to get caresian product of packed in a list or tuple. (note this deviates from api of
itertools.product()
)num (int | None) – maximum number of items to generate. If None generat them all
rng (int | float | None | numpy.random.RandomState | random.Random) – Seed or random number generator. Defaults to the global state of the python random module.
- Yields:
Tuple – a random item in the cartesian product
Example
>>> import ubelt as ub >>> items = [(1, 2, 3), (4, 5, 6, 7)] >>> rng = 0 >>> # xdoctest: +IGNORE_WANT >>> products = list(random_product(items, rng=0)) >>> print(ub.urepr(products, nl=0)) [(3, 4), (1, 7), (3, 6), (2, 7),... (1, 6), (2, 5), (2, 4)] >>> products = list(random_product(items, num=3, rng=0)) >>> print(ub.urepr(products, nl=0)) [(3, 4), (1, 7), (3, 6)]
Example
>>> # xdoctest: +REQUIRES(--profile) >>> rng = ensure_rng(0) >>> items = [np.array([15, 14]), np.array([27, 26]), >>> np.array([21, 22]), np.array([32, 31])] >>> num = 2 >>> for _ in range(100): >>> list(random_product(items, num=num, rng=rng))
- kwarray.util_random._npstate_to_pystate(npstate)[source]¶
Convert state of a NumPy RandomState object to a state that can be used by Python’s Random. Derived from [SO44313620].
References
Example
>>> py_rng = random.Random(0) >>> np_rng = np.random.RandomState(seed=0) >>> npstate = np_rng.get_state() >>> pystate = _npstate_to_pystate(npstate) >>> py_rng.setstate(pystate) >>> assert np_rng.rand() == py_rng.random()
- kwarray.util_random._pystate_to_npstate(pystate)[source]¶
Convert state of a Python Random object to state usable by NumPy RandomState. Derived from [SO44313620].
References
Example
>>> py_rng = random.Random(0) >>> np_rng = np.random.RandomState(seed=0) >>> pystate = py_rng.getstate() >>> npstate = _pystate_to_npstate(pystate) >>> np_rng.set_state(npstate) >>> assert np_rng.rand() == py_rng.random()
- kwarray.util_random._coerce_rng_type(rng)[source]¶
Internal method that transforms input seeds into an integer form.
- kwarray.util_random.ensure_rng(rng=None, api='numpy')[source]¶
Coerces input into a random number generator.
This function is useful for ensuring that your code uses a controlled internal random state that is independent of other modules.
If the input is None, then a global random state is returned.
If the input is a numeric value, then that is used as a seed to construct a random state.
If the input is a random number generator, then another random number generator with the same state is returned. Depending on the api, this random state is either return as-is, or used to construct an equivalent random state with the requested api.
- Parameters:
rng (int | float | None | numpy.random.RandomState | random.Random) – if None, then defaults to the global rng. Otherwise this can be an integer or a RandomState class. Defaults to the global random.
api (str) – specify the type of random number generator to use. This can either be ‘numpy’ for a
numpy.random.RandomState
object or ‘python’ for arandom.Random
object. Defaults to numpy.
- Returns:
rng - either a numpy or python random number generator, depending on the setting of
api
.- Return type:
Example
>>> rng = ensure_rng(None) >>> ensure_rng(0).randint(0, 1000) 684 >>> ensure_rng(np.random.RandomState(1)).randint(0, 1000) 37
Example
>>> num = 4 >>> print('--- Python as PYTHON ---') >>> py_rng = random.Random(0) >>> pp_nums = [py_rng.random() for _ in range(num)] >>> print(pp_nums) >>> print('--- Numpy as PYTHON ---') >>> np_rng = ensure_rng(random.Random(0), api='numpy') >>> np_nums = [np_rng.rand() for _ in range(num)] >>> print(np_nums) >>> print('--- Numpy as NUMPY---') >>> np_rng = np.random.RandomState(seed=0) >>> nn_nums = [np_rng.rand() for _ in range(num)] >>> print(nn_nums) >>> print('--- Python as NUMPY---') >>> py_rng = ensure_rng(np.random.RandomState(seed=0), api='python') >>> pn_nums = [py_rng.random() for _ in range(num)] >>> print(pn_nums) >>> assert np_nums == pp_nums >>> assert pn_nums == nn_nums
Example
>>> # Test that random modules can be coerced >>> import random >>> import numpy as np >>> ensure_rng(random, api='python') >>> ensure_rng(random, api='numpy') >>> ensure_rng(np.random, api='python') >>> ensure_rng(np.random, api='numpy')