BIOS Implementation Test Suite - news - bits-1146

bits-1146 released with these changes:

The "EFI, FFI, fo, fum" release.

BITS now uses the Python ctypes module to access many types of data structures and tables in memory, not just EFI. The ctypes module now works for non-EFI builds, so modules such as acpi and pirtable can safely use it. This required modifications to Python's libffi and ctypes modules, to handle the various required calling conventions. See the new bits.cdata module for ctypes wrappers that provide improved printing, formatting, bitfield handling, and types such as GUIDs.

Much of acpi, pirtable, mptable, and smbios now use ctypes instead of custom unpacking routines based on the struct module. Several new internal C routines provide access to data structures as addresses rather than Python strings, for direct access via ctypes. Further conversion to ctypes will allow significant reduction in C code in favor of Python.

Thanks to many new optimizations, BITS 1146 starts up several times faster than previous releases:

  • The filesystem API layer for Python code now has a faster implementation of stat(), reducing the cases where it needs to enumerate the parent directory to find the appropriate directory entry.

  • The Python pydoc module (with its various dependencies) now loads on-demand when first invoked via help() or the pydoc command.

  • Python now supports the zipimport module, and loads the Python standard library from a zipfile.

Other changes in this release:

  • Update Python to 2.7.9. Drop changes merged upstream, and add new stubs and compatibility APIs for functions now called by 2.7.9.

  • Update to ACPICA version 20141107

  • Support the Python shlex module

  • The pager now supports saving the currently displayed data to a file on disk. Since this requires file write support, it only works when booted via EFI.

  • Print a message when starting BITS and when loading and initializing early Python modules, to help debug BITS startup and provide an indication of progress.

  • smbios: Decode many additional structure types, and add a preliminary Annex A conformance test. Includes contributions by Cathy Ji.

  • efi: Add constants, structures, and helper functions for HII configuration protocols.

  • efi: The efi_file class now provides context manager support, to work with the "with" statement.

  • efi: Add menu entries to save and print EFI tables, in both raw and decoded form. Includes the System Table, Configuration Table, Runtime Services. and Boot Services. Decoded versions include names for known GUIDs in Configuration table entries.

  • acpi: Optionally allow I/O port access from ACPI method evaluation. By default, ACPI method evaluation still ignores attempts to read or write I/O ports. However, acpi.evaluate now accepts a keyword argument unsafe_io=True to explicitly allow I/O. Also add a new debug printf to show attempts to access I/O ports; add "acpica_io" to the debug environment variable to see those attempts.

  • acpi: When running under EFI, support saving ACPI tables to disk as separate files, both raw and optionally decoded.

  • acpi: Generate the table-dumping menu items from Python.

  • acpi: Support dumping decoded ACPI tables to the BITS log.

  • acpi: Add a new load_table method to load an SSDT table binary into the ACPI namespace. Usage: acpi.load_table(table_data), where table_data is a string buffer of the SSDT's AML. The contents of the SSDT can be used to either patch existing or add new ACPI methods or data for debug purposes. Given a binary AML file stored on the BITS boot media as /ssdt.aml: acpi.load_table(open("/ssdt.aml").read())

  • acpi: get_objpaths, display_objects, and dump now support limiting maximum object depth, with a new optional depth parameter.

  • acpica: Provide more logging and debugging functions, to help debug internal ACPICA failures.

  • _acpi: Add internal functions to get and set AcpiGbl_EnableAmlDebugObject

  • init.cfg: Add a commented example to enable verbose logging from Python; useful to debug Python module loading.

Bugfixes:

  • acpi: Fix double-free in _eval error path when processing packages

  • acpi: Handle package arguments correctly in acpi.evaluate. acpi.evaluate attempted to convert tuples to ACPI packages, but did not recursively convert the objects in the tuple to ACPI objects, so any attempt to pass a non-empty tuple as an argument failed. Fix by recursively calling _acpi_object_from_python on each component of the tuple.

  • acpi: Handle CPU paths and Device paths separately. Evaluate the former to get a ProcID, and call ._UID on the latter to get a UID; don't do both on both. Make display handle processors with UIDs correctly, and format them correctly.

  • acpimodule.c: Fix unused parameter warnings

    Previously hidden by lax warning flags that ACPICA requires; this will allow re-enabling that warning via GCC pragmas.

  • efi: import efi from generated menu entries; avoid assuming that some other global Python code has already imported efi.

  • efi: Make EFI table saving more robust (and verbose). Add error handling for unusual boot scenarios in which the EFI boot filesystem does not match the GRUB root directory.

  • Avoid printing spurious pre-import messages for efi modules when not on efi. Using try/except to catch ImportError didn't stop the initial message from printing. Check sys.platform instead.

  • ttypager: When trying to import efi, only catch ImportError

  • When viewing the log, don't write the log to the log