.tgt.heapyc.module heapyc

Name

module heapyc

Synopsis

Methods
HeapView(root: Any+, heapdefs: HeapDefs+) -> HeapView
interpreter(command: command+ [locals: dict+]) -> thread_id
NodeGraph( [edges: iterable+ [is_mapping: boolean+]]) -> NodeGraph
set_async_exc(thread_id: thread_id+, exception: Exception+)
xmemstats()
Attribute
RootState:  anything

Description

The heapyc module.

Methods

HeapView(root: Any+, heapdefs: HeapDefs+) -> HeapView
Returns a new HeapView object.
Arguments
root: Any+
The initial value of the root member.
heapdefs: HeapDefs+
Definitions of specially treated extension types.
interpreter(command: command+ [locals: dict+]) -> thread_id
Create a new interpreter structure with a new thread.
Arguments
command: command+
A command that will be exec'd in the new environment.
locals: dict+
Local variables passed to the command when exec'd.
Returns the thread identity number.
The new interpreter and thread is started in a new environment. This environment consists of a new '__main__' module, with the optional locals dict as local variables.
The site-specific initializations are not automatically made. To do that, 'import site' could be used as the first statement in the command string.
The interpreter() function will return after the new thread structure has been created. The command will execute sooner or later. The thread will terminate, and the interpreter structure be deallocated, when the command has been executed, and dependent threads have terminated.
NodeGraph( [edges: iterable+ [is_mapping: boolean+]]) -> NodeGraph
Construct a new NodeGraph object.
Arguments
edges: iterable+
The edges that will be used to initialize the new nodegraph. It should yield a sequence of pairs being edges of the form (source, target).
Default: ()
is_mapping: boolean+
If True, will cause the nodegraph to be treated like a 'mapping'. It will then, for the purpose of indexing, be expected to contain a single target for each source node.
Default: False
set_async_exc(thread_id: thread_id+, exception: Exception+)
Set an exception to be raised asynchronously in a thread.
xmemstats()
Print system-dependent memory statistics.
What is printed depends on the system configuration.

Attribute

RootState:  anything
The single instance of RootStateType.

.tgt.kindnames.HeapView

Name

HeapView

Synopsis

Constructor
module heapyc.HeapView(root: Any+, heapdefs: HeapDefs+) -> HeapView

For any object HV of kind HeapView:

Methods
HV.cli_class() -> ObjectClassifier
HV.cli_clodo(owners: NodeGraph+, memo: dict+) -> ObjectClassifier
HV.cli_indisize(memo: dict+) -> ObjectClassifier
HV.cli_none() -> ObjectClassifier
HV.cli_rcs(referrers: NodeGraph+, classifier: ObjectClassifier+, memo: dict+) -> ObjectClassifier
HV.cli_type() -> ObjectClassifier
HV.heap() -> NodeSet
HV.indisize_sum(S: iterable+) -> int
HV.numedges(src: Any+, tgt: Any+) -> int
HV.reachable(X: NodeSet+, Y: NodeSet+) -> NodeSet
HV.reachable_x(X: NodeSet+, Y: NodeSet+) -> NodeSet
HV.register__hiding_tag__type(type_with_hiding_tag+)
HV.register_hidden_exact_type(type+)
HV.relate(src: Any+, tgt: Any+) -> RelationStructure
HV.relimg(S: iterable+) -> NodeSet
HV.shpathstep(G: NodeGraph+, U: NodeSet+, S: NodeSet+ [AvoidEdges: NodeGraph+ [find_one: boolean+]]) -> NodeSet
HV.update_dictowners(owners: NodeGraph+)
HV.update_referrers(X: NodeGraph+, Y: NodeSet+)
HV.update_referrers_completely(X: NodeGraph+)
Attributes
HV._hiding_tag_:  anything
HV.delete_extra_type:  anything
HV.is_hiding_calling_interpreter: boolean
HV.is_using_traversing_owner_update: boolean
HV.limitframe:  (
either:[None or frame]
)

Description

A HeapView object provides methods to get memory related information about the system heap and about individual objects.

It implements much of the low-level functionality for the Heapy system. It is intended to provide what can not be done at all or would be much slower if programmed directly in Python. It is not intended to be used directly by a user, but to be wrapped in higher level objects.

Some terms that are referred to in the method descriptions:

Visible objects.

The HeapView object attempts to restrict its view of the heap to only the 'visible objects'. This is to make it possible to analyse the heap via a Python library that inevitably itself is continually allocating and deallocating objects. These should be hidden from the heap view presented. This is primarily done via a special tag attribute, see _hiding_tag_ and register__hiding_tag__type . Frames can be hidden with another mechanism, see limitframe. For hiding all objects of a special type, register_hidden_exact_type may be used. It is also possible to use a separate interpreter and hide its root objects, see is_hiding_calling_interpreter .

Classifiers.

The methods named cli_* are factory methods that create objects of type ObjectClassifier. The principal difference between classifiers is how a single object is classified. The single-object classification function is available in classifier objects; it is the classify method. There are also methods that operate on collections of objects, namely partition and select. These eliminate the per-object Python-level function call overhead that would occur if the classify method were to be called from Python for each object in a collection. See also the ObjectClassifier type.

Individual size.

The individual size of an object is its individually allocated memory size.

It includes:

  • The basic object size, as can be found out in a standard way.
  • The extra memory for variable size objects.
  • For GC collected objects, the size of the GC information.
  • An alignment to the next highest multiple of a pointer size.
  • The size of any other memory allocated that belongs to the object.

Some types of objects have extra memory allocated that can not be accounted for in the standard way. This memory should nevertheless be included in the individual size. To determine the size of these objects, special functions are needed. These are defined for standard builtin types, such as lists and dicts. Other types should be defined via the heapdefs argument to the HeapView constructor.

The individual size does not include:

  • Subobjects that are accounted for separately.
  • Overhead for the memory allocation system. This varies depending on the kind of memory allocator, the requested size, etc.

Methods

HV.cli_class() -> ObjectClassifier
Returns a classifier that classifies by "class".
If an object is of instance type, its classification will be its class, as given in its C-level member 'in_class', otherwise it will be its type, as given in 'ob_type'.
Note
This is like classifying by the Python-level '__class__' attribute in newer versions of Python, except that even though the '__class__' attribute may be redefined via the class definition, the classification as made by cli_class isn't affected.
HV.cli_clodo(owners: NodeGraph+, memo: dict+) -> ObjectClassifier
Returns a classifier, that classifies by "Class Or Dict Owner".
The classification of an object is its class (as by cli_class), unless the object is a dict object that is 'owned' by some owner. If the object is such an owned dict, the classification will be a 1-tuple containing the class (as by cli_class) of its owner.
Arguments
owners: NodeGraph+
Used to map each dict object to its owner, or to None if it has no owner. The graph will be automatically updated, from heap information defined by HV, whenever an attempt is made to classify a dict that maps to nothing.
memo: dict+
Used to memoize the 1-tuples generated when classifying owned dicts.
HV.cli_indisize(memo: dict+) -> ObjectClassifier
Returns a classifier that classifies by "individual size".
The classification of each object is an int, containing the object's individual memory size.
Argument
memo: dict+
Used to memoize the classification objects.
HV.cli_none() -> ObjectClassifier
Returns a classifier that classifies all objects the same.
The classification of each object is None.
HV.cli_rcs(referrers: NodeGraph+, classifier: ObjectClassifier+, memo: dict+) -> ObjectClassifier
Returns a classifier that classifies by Referrer Classification Set.
The classification of an object is the classifications of its referrers, collected in an immutable NodeSet object.
Arguments
referrers: NodeGraph+
Used to map each object to its referrers.
classifier: ObjectClassifier+
Used to classify each referrer.
memo: dict+
Used to memoize the classification sets.
HV.cli_type() -> ObjectClassifier
Returns a classifier that classifies by type.
The classification of each object is the type, as given by its C-level member 'ob_type'. (This is the same as the type returned by the Python-level builtin 'type'.)
HV.heap() -> NodeSet
Returns a set containing all 'visible objects' in the heap view defined by HV.
See also
Visible objects
HV.indisize_sum(S: iterable+) -> int
Returns the sum of the 'individual size' of the objects in S.
See also
Individual size
HV.numedges(src: Any+, tgt: Any+) -> int
Returns the number of edges from src to tgt.
HV.reachable(X: NodeSet+, Y: NodeSet+) -> NodeSet
Returns the set of objects reached via a path in the visible heap as defined by HV, from some object in X, avoiding any object in Y.
HV.reachable_x(X: NodeSet+, Y: NodeSet+) -> NodeSet
Returns the set of objects reached via a path in the visible heap as defined by HV, from some object in X, avoiding any object in Y except at the end of the path.
HV.register__hiding_tag__type(type_with_hiding_tag+)
Register a type of objects that may be hidden from the heap view defined by HV. The type must have a slot named _hiding_tag_. An object that is an instance of the type, or of a subtype, is hidden when its _hiding_tag_ is HV._hiding_tag_.
HV.register_hidden_exact_type(type+)
Register a type of objects that should be hidden from the heap view defined by HV. Objects of the exact type registered -- not including subtypes -- will be hidden.
HV.relate(src: Any+, tgt: Any+) -> RelationStructure
Returns a description of the relation between src and tgt.
This is used for descriptions of edges in paths.
[The result is in a special format that I choose to not define here since it is for special low-level use and subject to change.]
HV.relimg(S: iterable+) -> NodeSet
Returns the 'relational image of HV wrt S'.
That is, the set of nodes that are directly referred to from the nodes in S via the visible heap reachability relation as defined by HV.
HV.shpathstep(G: NodeGraph+, U: NodeSet+, S: NodeSet+ [AvoidEdges: NodeGraph+ [find_one: boolean+]]) -> NodeSet
This method implements one step of a shortest path algorithm.
Arguments
G: NodeGraph+
Updated by the method, with the edges from nodes in the source set to the new nodes visited.
U: NodeSet+
The source set for this step.
S: NodeSet+
The set of already visited nodes.
AvoidEdges: NodeGraph+
Edges to avoid.
find_one: boolean+
If True, at most one edge will be found from each node in the source set. Normally, all edges will be found.
Returns the new nodes visited.
This may be used for the U argument the next time the method is called.
See also
shpgraph_algorithm in Path.py.
HV.update_dictowners(owners: NodeGraph+)
Update owners with ownership edges.
The dict owners graph will be updated with an edge from each dict object in the heap, to either its owner or to None.
HV.update_referrers(X: NodeGraph+, Y: NodeSet+)
Update referrer graph X for Y.
The visible heap defined by HV will be traversed from the root of HV so that the edges of every path from the root to nodes in Y will be represented, inverted, in X.
HV.update_referrers_completely(X: NodeGraph+)
Update referrer graph X 'completely'.
[Experimental algorithm that updates X with the referrers to all objects in the heap (of visible nodes as defined in HV). It is not normally used.]

Attributes

HV._hiding_tag_:  anything
The hiding tag defining what objects are hidden from the view defined by HV. Objects that contain a _hiding_tag_ object which is identical to HV._hiding_tag_, will be hidden from view, in the following cases:
  • The object is of a type that has been registered for hiding via register__hiding_tag__type, or is of a subtype of such a type.
  • The object is of instance type. Such an object will be checked for a _hiding_tag_ item in its __dict__.
HV.delete_extra_type:  anything
For Internal Use
HV.is_hiding_calling_interpreter: boolean
If True, the data of the interpreter using the HV will be hidden from the heap view as seen from RootState.
This is used when multiple Python interpreters are used. One interpreter will be monitoring the operation of the other interpreter(s). It would set is_hiding_calling_interpreter to True in the HV it is using. Its own data will then be hidden from view, making memory leak detection more practical.
HV.is_using_traversing_owner_update: boolean
True if update_dictowners is using a recursive traversal algorithm to find the dicts in the heap. When False, the normal case, it will use the dicts found in the gc collection structure, by gc.get_objects(). This was found to be much faster in usual cases, but the old version is available by setting this flag. -- It may be removed in a later release!
HV.limitframe:  (
either:[None or frame]
)
The traversal limiting frame.
If limitframe is set to a frame object, the frames that are more recently entered than limitframe will be hidden when traversing the heap from the root RootState. It will start traversing from limitframe rather than from the most recent frame as it would otherwise do.

.tgt.kindnames.NodeGraph

Name

NodeGraph

Synopsis

Constructor
module heapyc.NodeGraph( [edges: iterable+ [is_mapping: boolean+]]) -> NodeGraph

For any object NG of kind NodeGraph:

Methods
NG.add_edge(source: Any+, target: Any+)
NG.add_edges_n1(srcs: iterable+, tgt: Any+)
NG.as_flat_list() -> list
NG.clear()
NG.copy() -> NodeGraph
NG.domain_covers(X: iterable+) -> boolean
NG.domain_restricted(X: iterable+) -> NodeGraph
NG.get_domain() -> NodeSet
NG.get_range() -> NodeSet
NG.invert()
NG.inverted() -> NodeGraph
NG.relimg(X: iterable+) -> NodeSet
NG.update(X: iterable+)
NG.updated(X: iterable+) -> NodeGraph
Operators
len(NG) -> int
# NG[source: Any+] -> Any
# NG[source: Any+] = target_spec: TargetSpec+
iter(NG) -> iterator
Attributes
NG._hiding_tag_:  anything
NG.is_mapping: boolean
NG.is_sorted: boolean

Description

A NodeGraph object contains pairs of nodes (edges) and can be indexed on the first node of such a pair (the source of an edge) to find all second nodes of such pairs (the targets of those edges).

NodeGraph objects are used internally in the Heapy system, for example to record dict ownership and shortest-path graphs.

They may be used generally for mapping and dict-like purposes, but differ from dicts in the following:

  • The mapping is based on object identity - no equality or hashing is assumed, so any object can be used as a key. Only the address is used. To distinguish this usage from that of ordinary dicts and sets, such objects are called 'nodes'.
  • There may be any number of targets associated with each source.
  • Performance characteristics differ from dicts, in somewhat subtle ways.

Methods

NG.add_edge(source: Any+, target: Any+)
Add to NG, an edge from source to target.
NG.add_edges_n1(srcs: iterable+, tgt: Any+)
Add to NG, for each src in srcs, an edge from src to tgt.
NG.as_flat_list() -> list
Returns the edges of NG in the form [src0, tgt0, src1, tgt1 ...].
NG.clear()
Remove all items from NG.
NG.copy() -> NodeGraph
Returns a copy of NG.
NG.domain_covers(X: iterable+) -> boolean
Returns True if each node in X is the source of some edge in NG, False otherwise.
NG.domain_restricted(X: iterable+) -> NodeGraph
Returns a new NodeGraph, containing those edges in NG that have the source in X.
NG.get_domain() -> NodeSet
Returns the set of nodes that are the source of some edge in NG.
NG.get_range() -> NodeSet
Returns the set of nodes that are the target of some edge in NG.
NG.invert()
Invert the edges of NG.
NG.inverted() -> NodeGraph
Returns a copy of NG with the edges inverted.
NG.relimg(X: iterable+) -> NodeSet
Returns the relational image of NG wrt X.
That is, the set of nodes that are the target of some edge that have its source in X.
NG.update(X: iterable+)
Update NG with the edges from X, specified as pairs of the form (source, target).
NG.updated(X: iterable+) -> NodeGraph
Returns a copy of NG updated with the edges from X, specified as pairs of the form (source, target).

Operators

len(NG) -> int
Returns the number of edges in NG.
NG[source: Any+] -> Any
Returns the target(s) of all edges with a particular source. The value depends on if NG was initialized to be a 'mapping' or not:
NG.is_mapping == False
Return a tuple of all targets of edges from the source. The tuple will be empty if there are no such edges.
NG.is_mapping == True
Return the target of the edge from the source. If there is no such edge, KeyError will be raised. If there is more than one edge, ValueError will be raised.
NG[source: Any+] = target_spec: TargetSpec+
Assign new target(s) to the edge(s) from a given source. How the targets are specified, depends on whether NG is a 'mapping' or not:
NG.is_mapping == False
The target_spec argument is a tuple of targets. There must already be the same number of edges in NG from the given source as the number of objects in target_spec.
NG.is_mapping == True
The target_spec argument is the target itself. There must already be exactly one edge in NG from the given source.
If there were not already the same number of edges as the targets specified, ValueError will be raised. The reason for the restriction is to make sure assignment will be efficient when interleaved with indexing.
See also
add_edge, add_edges_n1
iter(NG) -> iterator
Returns an iterator yielding a pair (source, target) for each edge in NG.

Attributes

NG._hiding_tag_:  anything
The hiding tag: if it is the the same object as HeapView._hiding_tag_ of a HeapView object, the nodegraph will be hidden from the corresponding heap view .
NG.is_mapping: boolean
Read only. True if NG is a 'mapping'. Then, only one edge is allowed for each source; indexing returns the actual target object instead of a tuple of targets.
NG.is_sorted: boolean
Read only. True if NG is sorted.
It will become unsorted after any update. It will need to be sorted to make it possible to find edges (implementation uses binary search). Any indexing operation will automatically sort it if it was not already sorted. The flag is currently used from Python to see if the nodegraph has been used at least once after update, so that it will not be cleared too early.

Generated by GSL-HTML 0.1.11 on Tue Sep 10 07:16:57 2019