'''"Executable documentation" for the pickle module.

Extensive comments about the pickle protocols and pickle-machine opcodes

can be found here. Some functions meant for external use:

genops(pickle)

Generate all the opcodes in a pickle, as (opcode, arg, position) triples.

dis(pickle, out=None, memo=None, indentlevel=4)

Print a symbolic disassembly of a pickle.

'''

__all__ = ['dis', 'genops', 'optimize']

# Other ideas:

#

# - A pickle verifier: read a pickle and check it exhaustively for

# well-formedness. dis() does a lot of this already.

#

# - A protocol identifier: examine a pickle and return its protocol number

# (== the highest .proto attr value among all the opcodes in the pickle).

# dis() already prints this info at the end.

#

# - A pickle optimizer: for example, tuple-building code is sometimes more

# elaborate than necessary, catering for the possibility that the tuple

# is recursive. Or lots of times a PUT is generated that's never accessed

# by a later GET.

"""

"A pickle" is a program for a virtual pickle machine (PM, but more accurately

called an unpickling machine). It's a sequence of opcodes, interpreted by the

PM, building an arbitrarily complex Python object.

For the most part, the PM is very simple: there are no looping, testing, or

conditional instructions, no arithmetic and no function calls. Opcodes are

executed once each, from first to last, until a STOP opcode is reached.

The PM has two data areas, "the stack" and "the memo".

Many opcodes push Python objects onto the stack; e.g., INT pushes a Python

integer object on the stack, whose value is gotten from a decimal string

literal immediately following the INT opcode in the pickle bytestream. Other

opcodes take Python objects off the stack. The result of unpickling is

whatever object is left on the stack when the final STOP opcode is executed.

The memo is simply an array of objects, or it can be implemented as a dict

mapping little integers to objects. The memo serves as the PM's "long term

memory", and the little integers indexing the memo are akin to variable

names. Some opcodes pop a stack object into the memo at a given index,

and others push a memo object at a given index onto the stack again.

At heart, that's all the PM has. Subtleties arise for these reasons:

+ Object identity. Objects can be arbitrarily complex, and subobjects

may be shared (for example, the list [a, a] refers to the same object a

twice). It can be vital that unpickling recreate an isomorphic object

graph, faithfully reproducing sharing.

+ Recursive objects. For example, after "L = []; L.append(L)", L is a

list, and L[0] is the same list. This is related to the object identity

point, and some sequences of pickle opcodes are subtle in order to

get the right result in all cases.

+ Things pickle doesn't know everything about. Examples of things pickle

does know everything about are Python's builtin scalar and container

types, like ints and tuples. They generally have opcodes dedicated to

them. For things like module references and instances of user-defined

classes, pickle's knowledge is limited. Historically, many enhancements

have been made to the pickle protocol in order to do a better (faster,

and/or more compact) job on those.

+ Backward compatibility and micro-optimization. As explained below,

pickle opcodes never go away, not even when better ways to do a thing

get invented. The repertoire of the PM just keeps growing over time.

For example, protocol 0 had two opcodes for building Python integers (INT

and LONG), protocol 1 added three more for more-efficient pickling of short

integers, and protocol 2 added two more for more-efficient pickling of

long integers (before protocol 2, the only ways to pickle a Python long

took time quadratic in the number of digits, for both pickling and

unpickling). "Opcode bloat" isn't so much a subtlety as a source of

wearying complication.

Pickle protocols:

For compatibility, the meaning of a pickle opcode never changes. Instead new

pickle opcodes get added, and each version's unpickler can handle all the

pickle opcodes in all protocol versions to date. So old pickles continue to

be readable forever. The pickler can generally be told to restrict itself to

the subset of opcodes available under previous protocol versions too, so that

users can create pickles under the current version readable by older

versions. However, a pickle does not contain its version number embedded

within it. If an older unpickler tries to read a pickle using a later

protocol, the result is most likely an exception due to seeing an unknown (in

the older unpickler) opcode.

The original pickle used what's now called "protocol 0", and what was called

"text mode" before Python 2.3. The entire pickle bytestream is made up of

printable 7-bit ASCII characters, plus the newline character, in protocol 0.

That's why it was called text mode. Protocol 0 is small and elegant, but

sometimes painfully inefficient.

The second major set of additions is now called "protocol 1", and was called

"binary mode" before Python 2.3. This added many opcodes with arguments

consisting of arbitrary bytes, including NUL bytes and unprintable "high bit"

bytes. Binary mode pickles can be substantially smaller than equivalent

text mode pickles, and sometimes faster too; e.g., BININT represents a 4-byte

int as 4 bytes following the opcode, which is cheaper to unpickle than the

(perhaps) 11-character decimal string attached to INT. Protocol 1 also added

a number of opcodes that operate on many stack elements at once (like APPENDS

and SETITEMS), and "shortcut" opcodes (like EMPTY_DICT and EMPTY_TUPLE).

The third major set of additions came in Python 2.3, and is called "protocol

2". This added:

- A better way to pickle instances of new-style classes (NEWOBJ).

- A way for a pickle to identify its protocol (PROTO).

- Time- and space- efficient pickling of long ints (LONG{1,4}).

- Shortcuts for small tuples (TUPLE{1,2,3}}.

- Dedicated opcodes for bools (NEWTRUE, NEWFALSE).

- The "extension registry", a vector of popular objects that can be pushed

efficiently by index (EXT{1,2,4}). This is akin to the memo and GET, but

the registry contents are predefined (there's nothing akin to the memo's

PUT).

Another independent change with Python 2.3 is the abandonment of any

pretense that it might be safe to load pickles received from untrusted

parties -- no sufficient security analysis has been done to guarantee

this and there isn't a use case that warrants the expense of such an

analysis.

To this end, all tests for __safe_for_unpickling__ or for

copy_reg.safe_constructors are removed from the unpickling code.

References to these variables in the descriptions below are to be seen

as describing unpickling in Python 2.2 and before.

"""

# Meta-rule: Descriptions are stored in instances of descriptor objects,

# with plain constructors. No meta-language is defined from which

# descriptors could be constructed. If you want, e.g., XML, write a little

# program to generate XML from the objects.

##############################################################################

# Some pickle opcodes have an argument, following the opcode in the

# bytestream. An argument is of a specific type, described by an instance

# of ArgumentDescriptor. These are not to be confused with arguments taken

# off the stack -- ArgumentDescriptor applies only to arguments embedded in

# the opcode stream, immediately following an opcode.

# Represents the number of bytes consumed by an argument delimited by the

# next newline character.

UP_TO_NEWLINE = -1

# Represents the number of bytes consumed by a two-argument opcode where

# the first argument gives the number of bytes in the second argument.

TAKEN_FROM_ARGUMENT1 = -2 # num bytes is 1-byte unsigned int

TAKEN_FROM_ARGUMENT4 = -3 # num bytes is 4-byte signed little-endian int

class ArgumentDescriptor(object):

__slots__ = (

# name of descriptor record, also a module global name; a string

'name',

# length of argument, in bytes; an int; UP_TO_NEWLINE and

# TAKEN_FROM_ARGUMENT{1,4} are negative values for variable-length

# cases

'n',

# a function taking a file-like object, reading this kind of argument

# from the object at the current position, advancing the current

# position by n bytes, and returning the value of the argument

'reader',

# human-readable docs for this arg descriptor; a string

'doc',

)

def __init__(self, name, n, reader, doc):

assert isinstance(name, str)

self.name = name

assert isinstance(n, (int, long)) and (n >= 0 or

n in (UP_TO_NEWLINE,

TAKEN_FROM_ARGUMENT1,

TAKEN_FROM_ARGUMENT4))

self.n = n

self.reader = reader

assert isinstance(doc, str)

self.doc = doc

from struct import unpack as _unpack

def read_uint1(f):

r"""

>>> import StringIO

>>> read_uint1(StringIO.StringIO('\xff'))

255

"""

data = f.read(1)

if data:

return ord(data)

raise ValueError("not enough data in stream to read uint1")

uint1 = ArgumentDescriptor(

name='uint1',

n=1,

reader=read_uint1,

doc="One-byte unsigned integer.")

def read_uint2(f):

r"""

>>> import StringIO

>>> read_uint2(StringIO.StringIO('\xff\x00'))

255

>>> read_uint2(StringIO.StringIO('\xff\xff'))

65535

"""

data = f.read(2)

if len(data) == 2:

return _unpack("<H", data)[0]

raise ValueError("not enough data in stream to read uint2")

uint2 = ArgumentDescriptor(

name='uint2',

n=2,

reader=read_uint2,

doc="Two-byte unsigned integer, little-endian.")

def read_int4(f):

r"""

>>> import StringIO

>>> read_int4(StringIO.StringIO('\xff\x00\x00\x00'))

255

>>> read_int4(StringIO.StringIO('\x00\x00\x00\x80')) == -(2**31)

True

"""

data = f.read(4)

if len(data) == 4:

return _unpack("<i", data)[0]

raise ValueError("not enough data in stream to read int4")

int4 = ArgumentDescriptor(

name='int4',

n=4,

reader=read_int4,

doc="Four-byte signed integer, little-endian, 2's complement.")

def read_stringnl(f, decode=True, stripquotes=True):

r"""

>>> import StringIO

>>> read_stringnl(StringIO.StringIO("'abcd'\nefg\n"))

'abcd'

>>> read_stringnl(StringIO.StringIO("\n"))

Traceback (most recent call last):

...

ValueError: no string quotes around ''

>>> read_stringnl(StringIO.StringIO("\n"), stripquotes=False)

''

>>> read_stringnl(StringIO.StringIO("''\n"))

''

>>> read_stringnl(StringIO.StringIO('"abcd"'))

Traceback (most recent call last):

...

ValueError: no newline found when trying to read stringnl

Embedded escapes are undone in the result.

>>> read_stringnl(StringIO.StringIO(r"'a\n\\b\x00c\td'" + "\n'e'"))

'a\n\\b\x00c\td'

"""

data = f.readline()

if not data.endswith('\n'):

raise ValueError("no newline found when trying to read stringnl")

data = data[:-1] # lose the newline

if stripquotes:

for q in "'\"":

if data.startswith(q):

if not data.endswith(q):

raise ValueError("strinq quote %r not found at both "

"ends of %r" % (q, data))

data = data[1:-1]

break

else:

raise ValueError("no string quotes around %r" % data)

# I'm not sure when 'string_escape' was added to the std codecs; it's

# crazy not to use it if it's there.

if decode:

data = data.decode('string_escape')

return data

stringnl = ArgumentDescriptor(

name='stringnl',

n=UP_TO_NEWLINE,

reader=read_stringnl,

doc="""A newline-terminated string.

This is a repr-style string, with embedded escapes, and

bracketing quotes.

""")

def read_stringnl_noescape(f):

return read_stringnl(f, decode=False, stripquotes=False)

stringnl_noescape = ArgumentDescriptor(

name='stringnl_noescape',

n=UP_TO_NEWLINE,

reader=read_stringnl_noescape,

doc="""A newline-terminated string.

This is a str-style string, without embedded escapes,

or bracketing quotes. It should consist solely of

printable ASCII characters.

""")

def read_stringnl_noescape_pair(f):

r"""

>>> import StringIO

>>> read_stringnl_noescape_pair(StringIO.StringIO("Queue\nEmpty\njunk"))

'Queue Empty'

"""

return "%s %s" % (read_stringnl_noescape(f), read_stringnl_noescape(f))

stringnl_noescape_pair = ArgumentDescriptor(

name='stringnl_noescape_pair',

n=UP_TO_NEWLINE,

reader=read_stringnl_noescape_pair,

doc="""A pair of newline-terminated strings.

These are str-style strings, without embedded

escapes, or bracketing quotes. They should

consist solely of printable ASCII characters.

The pair is returned as a single string, with

a single blank separating the two strings.

""")

def read_string4(f):

r"""

>>> import StringIO

>>> read_string4(StringIO.StringIO("\x00\x00\x00\x00abc"))

''

>>> read_string4(StringIO.StringIO("\x03\x00\x00\x00abcdef"))

'abc'

>>> read_string4(StringIO.StringIO("\x00\x00\x00\x03abcdef"))

Traceback (most recent call last):

...

ValueError: expected 50331648 bytes in a string4, but only 6 remain

"""

n = read_int4(f)

if n < 0:

raise ValueError("string4 byte count < 0: %d" % n)

data = f.read(n)

if len(data) == n:

return data

raise ValueError("expected %d bytes in a string4, but only %d remain" %

(n, len(data)))

string4 = ArgumentDescriptor(

name="string4",

n=TAKEN_FROM_ARGUMENT4,

reader=read_string4,

doc="""A counted string.

The first argument is a 4-byte little-endian signed int giving

the number of bytes in the string, and the second argument is

that many bytes.

""")

def read_string1(f):

r"""

>>> import StringIO

>>> read_string1(StringIO.StringIO("\x00"))

''

>>> read_string1(StringIO.StringIO("\x03abcdef"))

'abc'

"""

n = read_uint1(f)

assert n >= 0

data = f.read(n)

if len(data) == n:

return data

raise ValueError("expected %d bytes in a string1, but only %d remain" %

(n, len(data)))

string1 = ArgumentDescriptor(

name="string1",

n=TAKEN_FROM_ARGUMENT1,

reader=read_string1,

doc="""A counted string.

The first argument is a 1-byte unsigned int giving the number

of bytes in the string, and the second argument is that many

bytes.

""")

def read_unicodestringnl(f):

r"""

>>> import StringIO

>>> read_unicodestringnl(StringIO.StringIO("abc\uabcd\njunk"))

u'abc\uabcd'

"""

data = f.readline()

if not data.endswith('\n'):

raise ValueError("no newline found when trying to read "

"unicodestringnl")

data = data[:-1] # lose the newline

return unicode(data, 'raw-unicode-escape')

unicodestringnl = ArgumentDescriptor(

name='unicodestringnl',

n=UP_TO_NEWLINE,

reader=read_unicodestringnl,

doc="""A newline-terminated Unicode string.

This is raw-unicode-escape encoded, so consists of

printable ASCII characters, and may contain embedded

escape sequences.

""")

def read_unicodestring4(f):

r"""

>>> import StringIO

>>> s = u'abcd\uabcd'

>>> enc = s.encode('utf-8')

>>> enc

'abcd\xea\xaf\x8d'

>>> n = chr(len(enc)) + chr(0) * 3 # little-endian 4-byte length

>>> t = read_unicodestring4(StringIO.StringIO(n + enc + 'junk'))

>>> s == t

True

>>> read_unicodestring4(StringIO.StringIO(n + enc[:-1]))

Traceback (most recent call last):

...

ValueError: expected 7 bytes in a unicodestring4, but only 6 remain

"""

n = read_int4(f)

if n < 0:

raise ValueError("unicodestring4 byte count < 0: %d" % n)

data = f.read(n)

if len(data) == n:

return unicode(data, 'utf-8')

raise ValueError("expected %d bytes in a unicodestring4, but only %d "

"remain" % (n, len(data)))

unicodestring4 = ArgumentDescriptor(

name="unicodestring4",

n=TAKEN_FROM_ARGUMENT4,

reader=read_unicodestring4,

doc="""A counted Unicode string.

The first argument is a 4-byte little-endian signed int

giving the number of bytes in the string, and the second

argument-- the UTF-8 encoding of the Unicode string --

contains that many bytes.

""")

def read_decimalnl_short(f):

r"""

>>> import StringIO

>>> read_decimalnl_short(StringIO.StringIO("1234\n56"))

1234

>>> read_decimalnl_short(StringIO.StringIO("1234L\n56"))

Traceback (most recent call last):

...

ValueError: trailing 'L' not allowed in '1234L'

"""

s = read_stringnl(f, decode=False, stripquotes=False)

if s.endswith("L"):

raise ValueError("trailing 'L' not allowed in %r" % s)

# It's not necessarily true that the result fits in a Python short int:

# the pickle may have been written on a 64-bit box. There's also a hack

# for True and False here.

if s == "00":

return False

elif s == "01":

return True

try:

return int(s)

except OverflowError:

return long(s)

def read_decimalnl_long(f):

r"""

>>> import StringIO

>>> read_decimalnl_long(StringIO.StringIO("1234\n56"))

Traceback (most recent call last):

...

ValueError: trailing 'L' required in '1234'

Someday the trailing 'L' will probably go away from this output.

>>> read_decimalnl_long(StringIO.StringIO("1234L\n56"))

1234L

>>> read_decimalnl_long(StringIO.StringIO("123456789012345678901234L\n6"))

123456789012345678901234L

"""

s = read_stringnl(f, decode=False, stripquotes=False)

if not s.endswith("L"):

raise ValueError("trailing 'L' required in %r" % s)

return long(s)

decimalnl_short = ArgumentDescriptor(

name='decimalnl_short',

n=UP_TO_NEWLINE,

reader=read_decimalnl_short,

doc="""A newline-terminated decimal integer literal.

This never has a trailing 'L', and the integer fit

in a short Python int on the box where the pickle

was written -- but there's no guarantee it will fit

in a short Python int on the box where the pickle

is read.

""")

decimalnl_long = ArgumentDescriptor(

name='decimalnl_long',

n=UP_TO_NEWLINE,

reader=read_decimalnl_long,

doc="""A newline-terminated decimal integer literal.

This has a trailing 'L', and can represent integers

of any size.

""")

def read_floatnl(f):

r"""

>>> import StringIO

>>> read_floatnl(StringIO.StringIO("-1.25\n6"))

-1.25

"""

s = read_stringnl(f, decode=False, stripquotes=False)

return float(s)

floatnl = ArgumentDescriptor(

name='floatnl',

n=UP_TO_NEWLINE,

reader=read_floatnl,

doc="""A newline-terminated decimal floating literal.

In general this requires 17 significant digits for roundtrip

identity, and pickling then unpickling infinities, NaNs, and

minus zero doesn't work across boxes, or on some boxes even

on itself (e.g., Windows can't read the strings it produces

for infinities or NaNs).

""")

def read_float8(f):

r"""

>>> import StringIO, struct

>>> raw = struct.pack(">d", -1.25)

>>> raw

'\xbf\xf4\x00\x00\x00\x00\x00\x00'

>>> read_float8(StringIO.StringIO(raw + "\n"))

-1.25

"""

data = f.read(8)

if len(data) == 8:

return _unpack(">d", data)[0]

raise ValueError("not enough data in stream to read float8")

float8 = ArgumentDescriptor(

name='float8',

n=8,

reader=read_float8,

doc="""An 8-byte binary representation of a float, big-endian.

The format is unique to Python, and shared with the struct

module (format string '>d') "in theory" (the struct and cPickle

implementations don't share the code -- they should). It's

strongly related to the IEEE-754 double format, and, in normal

cases, is in fact identical to the big-endian 754 double format.

On other boxes the dynamic range is limited to that of a 754

double, and "add a half and chop" rounding is used to reduce

the precision to 53 bits. However, even on a 754 box,

infinities, NaNs, and minus zero may not be handled correctly

(may not survive roundtrip pickling intact).

""")

# Protocol 2 formats

from pickle import decode_long

def read_long1(f):

r"""

>>> import StringIO

>>> read_long1(StringIO.StringIO("\x00"))

0L

>>> read_long1(StringIO.StringIO("\x02\xff\x00"))

255L

>>> read_long1(StringIO.StringIO("\x02\xff\x7f"))

32767L

>>> read_long1(StringIO.StringIO("\x02\x00\xff"))

-256L

>>> read_long1(StringIO.StringIO("\x02\x00\x80"))

-32768L

"""

n = read_uint1(f)

data = f.read(n)

if len(data) != n:

raise ValueError("not enough data in stream to read long1")

return decode_long(data)

long1 = ArgumentDescriptor(

name="long1",

n=TAKEN_FROM_ARGUMENT1,

reader=read_long1,

doc="""A binary long, little-endian, using 1-byte size.

This first reads one byte as an unsigned size, then reads that

many bytes and interprets them as a little-endian 2's-complement long.

If the size is 0, that's taken as a shortcut for the long 0L.

""")

def read_long4(f):

r"""

>>> import StringIO

>>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x00"))

255L

>>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x7f"))

32767L

>>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\xff"))

-256L

>>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\x80"))

-32768L

>>> read_long1(StringIO.StringIO("\x00\x00\x00\x00"))

0L

"""

n = read_int4(f)

if n < 0:

raise ValueError("long4 byte count < 0: %d" % n)

data = f.read(n)

if len(data) != n:

raise ValueError("not enough data in stream to read long4")

return decode_long(data)

long4 = ArgumentDescriptor(

name="long4",

n=TAKEN_FROM_ARGUMENT4,

reader=read_long4,

doc="""A binary representation of a long, little-endian.

This first reads four bytes as a signed size (but requires the

size to be >= 0), then reads that many bytes and interprets them

as a little-endian 2's-complement long. If the size is 0, that's taken

as a shortcut for the long 0L, although LONG1 should really be used

then instead (and in any case where # of bytes < 256).

""")

##############################################################################

# Object descriptors. The stack used by the pickle machine holds objects,

# and in the stack_before and stack_after attributes of OpcodeInfo

# descriptors we need names to describe the various types of objects that can

# appear on the stack.

class StackObject(object):

__slots__ = (

# name of descriptor record, for info only

'name',

# type of object, or tuple of type objects (meaning the object can

# be of any type in the tuple)

'obtype',

# human-readable docs for this kind of stack object; a string

'doc',

)

def __init__(self, name, obtype, doc):

assert isinstance(name, str)

self.name = name

assert isinstance(obtype, type) or isinstance(obtype, tuple)

if isinstance(obtype, tuple):

for contained in obtype:

assert isinstance(contained, type)

self.obtype = obtype

assert isinstance(doc, str)

self.doc = doc

def __repr__(self):

return self.name

pyint = StackObject(

name='int',

obtype=int,

doc="A short (as opposed to long) Python integer object.")

pylong = StackObject(

name='long',

obtype=long,

doc="A long (as opposed to short) Python integer object.")

pyinteger_or_bool = StackObject(

name='int_or_bool',

obtype=(int, long, bool),

doc="A Python integer object (short or long), or "

"a Python bool.")

pybool = StackObject(

name='bool',

obtype=(bool,),

doc="A Python bool object.")

pyfloat = StackObject(

name='float',

obtype=float,

doc="A Python float object.")

pystring = StackObject(

name='str',

obtype=str,

doc="A Python string object.")

pyunicode = StackObject(

name='unicode',

obtype=unicode,

doc="A Python Unicode string object.")

pynone = StackObject(

name="None",

obtype=type(None),

doc="The Python None object.")

pytuple = StackObject(

name="tuple",

obtype=tuple,

doc="A Python tuple object.")

pylist = StackObject(

name="list",

obtype=list,

doc="A Python list object.")

pydict = StackObject(

name="dict",

obtype=dict,

doc="A Python dict object.")

anyobject = StackObject(

name='any',

obtype=object,

doc="Any kind of object whatsoever.")

markobject = StackObject(

name="mark",

obtype=StackObject,

doc="""'The mark' is a unique object.

Opcodes that operate on a variable number of objects

generally don't embed the count of objects in the opcode,

or pull it off the stack. Instead the MARK opcode is used

to push a special marker object on the stack, and then

some other opcodes grab all the objects from the top of

the stack down to (but not including) the topmost marker

object.

""")

stackslice = StackObject(

name="stackslice",

obtype=StackObject,

doc="""An object representing a contiguous slice of the stack.

This is used in conjunction with markobject, to represent all

of the stack following the topmost markobject. For example,

the POP_MARK opcode changes the stack from

[..., markobject, stackslice]

to

[...]

No matter how many object are on the stack after the topmost

markobject, POP_MARK gets rid of all of them (including the

topmost markobject too).

""")

##############################################################################

# Descriptors for pickle opcodes.

class OpcodeInfo(object):

__slots__ = (

# symbolic name of opcode; a string

'name',

# the code used in a bytestream to represent the opcode; a

# one-character string

'code',

# If the opcode has an argument embedded in the byte string, an

# instance of ArgumentDescriptor specifying its type. Note that

# arg.reader(s) can be used to read and decode the argument from

# the bytestream s, and arg.doc documents the format of the raw

# argument bytes. If the opcode doesn't have an argument embedded

# in the bytestream, arg should be None.

'arg',

# what the stack looks like before this opcode runs; a list

'stack_before',

# what the stack looks like after this opcode runs; a list

'stack_after',

# the protocol number in which this opcode was introduced; an int

'proto',

# human-readable docs for this opcode; a string

'doc',

)

def __init__(self, name, code, arg,

stack_before, stack_after, proto, doc):

assert isinstance(name, str)

self.name = name

assert isinstance(code, str)

assert len(code) == 1

self.code = code

assert arg is None or isinstance(arg, ArgumentDescriptor)

self.arg = arg

assert isinstance(stack_before, list)

for x in stack_before:

assert isinstance(x, StackObject)

self.stack_before = stack_before

assert isinstance(stack_after, list)

for x in stack_after:

assert isinstance(x, StackObject)

self.stack_after = stack_after

assert isinstance(proto, (int, long)) and 0 <= proto <= 2

self.proto = proto

assert isinstance(doc, str)

self.doc = doc

I = OpcodeInfo

opcodes = [

# Ways to spell integers.

I(name='INT',

code='I',

arg=decimalnl_short,

stack_before=[],

stack_after=[pyinteger_or_bool],

proto=0,

doc="""Push an integer or bool.

The argument is a newline-terminated decimal literal string.

The intent may have been that this always fit in a short Python int,

but INT can be generated in pickles written on a 64-bit box that

require a Python long on a 32-bit box. The difference between this

and LONG then is that INT skips a trailing 'L', and produces a short

int whenever possible.

Another difference is due to that, when bool was introduced as a

distinct type in 2.3, builtin names True and False were also added to

2.2.2, mapping to ints 1 and 0. For compatibility in both directions,

True gets pickled as INT + "I01\\n", and False as INT + "I00\\n".

Leading zeroes are never produced for a genuine integer. The 2.3

(and later) unpicklers special-case these and return bool instead;

earlier unpicklers ignore the leading "0" and return the int.

"""),

I(name='BININT',

code='J',

arg=int4,

stack_before=[],

stack_after=[pyint],

proto=1,

doc="""Push a four-byte signed integer.

This handles the full range of Python (short) integers on a 32-bit

box, directly as binary bytes (1 for the opcode and 4 for the integer).

If the integer is non-negative and fits in 1 or 2 bytes, pickling via

BININT1 or BININT2 saves space.

"""),

I(name='BININT1',

code='K',

arg=uint1,

stack_before=[],

stack_after=[pyint],

proto=1,

doc="""Push a one-byte unsigned integer.

This is a space optimization for pickling very small non-negative ints,

in range(256).

"""),

I(name='BININT2',

code='M',

arg=uint2,

stack_before=[],

stack_after=[pyint],

proto=1,

doc="""Push a two-byte unsigned integer.

This is a space optimization for pickling small positive ints, in

range(256, 2**16). Integers in range(256) can also be pickled via

BININT2, but BININT1 instead saves a byte.

"""),

I(name='LONG',

code='L',

arg=decimalnl_long,

stack_before=[],

stack_after=[pylong],

proto=0,

doc="""Push a long integer.

The same as INT, except that the literal ends with 'L', and always

unpickles to a Python long. There doesn't seem a real purpose to the

trailing 'L'.

Note that LONG takes time quadratic in the number of digits when

unpickling (this is simply due to the nature of decimal->binary

conversion). Proto 2 added linear-time (in C; still quadratic-time

in Python) LONG1 and LONG4 opcodes.

"""),

I(name="LONG1",

code='\x8a',

arg=long1,

stack_before=[],

stack_after=[pylong],

proto=2,

doc="""Long integer using one-byte length.

A more efficient encoding of a Python long; the long1 encoding

says it all."""),

I(name="LONG4",

code='\x8b',

arg=long4,

stack_before=[],

stack_after=[pylong],

proto=2,

doc="""Long integer using found-byte length.

A more efficient encoding of a Python long; the long4 encoding

says it all."""),

# Ways to spell strings (8-bit, not Unicode).

I(name='STRING',

code='S',

arg=stringnl,

stack_before=[],

stack_after=[pystring],

proto=0,

doc="""Push a Python string object.