misc: Merge branch 'release-staging-v22-0' into stable
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index bdbcc2d..189b63f 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -144,13 +144,17 @@
A canonical commit message consists of three parts:
* A short summary line describing the change. This line starts with one or
more keywords (found in the MAINTAINERS file) separated by commas followed
- by a colon and a description of the change. This line should be no more than
- 65 characters long since version control systems usually add a prefix that
- causes line-wrapping for longer lines.
+ by a colon and a description of the change. This short description is
+ written in the imperative mood, and should say what happens when the patch
+ is applied. Keep it short and simple. Write it in sentence case preferably
+ not ending in a period. This line should be no more than 65 characters long
+ since version control systems usually add a prefix that causes line-wrapping
+ for longer lines.
* (Optional, but highly recommended) A detailed description. This describes
what you have done and why. If the change isn't obvious, you might want to
motivate why it is needed. Lines need to be wrapped to 72 characters or
- less.
+ less. Leave a blank line between the first short summary line and this
+ detailed description.
* Tags describing patch metadata. You are highly recommended to use
tags to acknowledge reviewers for their work. Gerrit will automatically add
most tags.
diff --git a/KCONFIG.md b/KCONFIG.md
new file mode 100644
index 0000000..9292539
--- /dev/null
+++ b/KCONFIG.md
@@ -0,0 +1,114 @@
+This file explains how to work with gem5's implementation of the kconfig
+configuration system, very similar to what's used by the linux kernel. It talks
+about how to work with the Kconfig files themselves which define what user
+adjustable configuration parameters there are, and how they work and
+interoperate.
+
+This file does *not*:
+
+ * Describe how kconfig works generally. This is well documented elsewhere, for
+ instance [here](
+ https://www.kernel.org/doc/html/latest/kbuild/kconfig-language.html):
+ * The settings in gem5's kconfig files. These should be documented with help
+ text within the kconfig files, and in the code where these options are
+ consumed.
+ * The various tools which can manipulate a given configuration. These are
+ documented in gem5's SCons help text.
+
+# kconfiglib
+
+gem5 uses the kconfiglib python library for consuming and manipulating kconfig
+configurations. It is very similar to the kconfig implementation used by the
+linux kernel, but is slightly different and has some small extensions added to
+it. Almost all kconfig documentation written for the kernel's implementation
+should apply here as well, but it may occasionally be necessary to refer to the
+kconfiglib documentation.
+
+Also, because gem5's build system is more modular than kconfig's design
+supports out of the box, particularly for "choice" blocks, we have extended
+kconfiglib and added a "cont_choice" keyword. This keyword is very similar to
+"choice", except "cont_choice" blocks can be re-opened and extended with more
+options further into the config.
+
+This can be used to set up a central point where the user can choose between
+mutually exclusive options, and still allow new Kconfig files to add new
+options without modifying the original source.
+
+Having to choose between mutually exclusive options should be avoided in
+general, but is unavoidable in a few key places in gem5 at the moment. Once
+those areas have been addressed, this keyword may be removed in the future.
+
+# The 'CONF' dict in the SCons environment
+
+In "env" SCons environment in SConscript files, or the "main" environment in
+SConsopts files, can hold many variables which help SCons operate generally,
+like setting what include paths to use, what the compiler command line is, etc.
+These environments each have a 'CONF' sub-dict which holds all the variables
+which are actually used to configure gem5, and not to configure SCons and the
+build process itself.
+
+All variables in this dict are automatically available to include in c++. To
+access the value of env['CONF']['FOO'], you would #include "config/foo.hh".
+Because these variables are in a shared namespace, their names should be unique
+and distinctive.
+
+These values are available in config scripts through the m5.defines.buildEnv
+dict.
+
+# Automatic/measured configuration values.
+
+Some configuration values are not set by the user, and are measured from the
+host environment. These could reflect the availability of a header file,
+library or tool, whether the compiler supports a particular option or language
+feature, etc.
+
+These values should be measured in SConsopts files, and stored in the 'CONF'
+dict described above. Like any other variable in 'CONF', they are then
+available to C++ through generated header files, to config scripts through
+buildEnv, etc. They are also available in the kconfig files themselves through
+a mechanism discussed below.
+
+# Accessing 'CONF' values in Kconfig files.
+
+When the gem5 Kconfig files are processed to either manipulate a configuration
+through a tool, or to apply a configuration to the gem5 build, all the values
+in 'CONF' are temporarily put into environment variables. In the Kconfig files
+themselves, these environment variables can be accessed using $(FOO) syntax,
+which is described in kconfiglib's documentation.
+
+Note that this is slightly different from the kernel's Kconfig syntax, where
+the environment variables would have to be imported in using other keywords
+first.
+
+This is generally used to make automatic/measured settings which were
+determined in SConsopts files available in Kconfig files. They can then be used
+to compute dependencies, or to set default values, etc.
+
+# Structure of the Kconfig hierarchy
+
+Unlike SConscript files, gem5 does not find Kconfig files automatically, and
+they are only used if they are included explicitly in other Kconfig files.
+
+Kconfig options should be defined as close as possible to where they are used.
+This makes them easier to find, keeps related functionality grouped
+together in the source tree, and minimizes conflicts from modifying the same
+few, central files when changing unrelated parts of gem5.
+
+When including a Kconfig file in another, you should use the "rsource" keyword
+which is a kconfiglib extension. This lets you include the other file using a
+path which is relative to the current file, and also helps make the kconfig
+files more modular and self contained.
+
+# EXTRAS directories.
+
+The EXTRAS variable can be set to a list of directories which hold additional
+source that should be built into gem5. Because there's no way to know what (if
+any) paths will be in EXTRAS ahead of time, it is not possible to explicitly
+include kconfig files in those directories from a static file.
+
+Instead, gem5's real root Kconfig file, which includes the one in src, is
+automatically generated as part of the build. It uses the kconfiglib extension
+"osource" to optionally source a file called Kconfig in the base of each EXTRAS
+directory after it has sourced gem5's main Kconfig. If you want to add Kconfig
+options to your EXTRAS directory, you can create that file, and then rsource
+any additional internal Kconfig files as needed.
diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md
index 5d569b4..d199fed 100644
--- a/RELEASE-NOTES.md
+++ b/RELEASE-NOTES.md
@@ -1,3 +1,118 @@
+# Version 22.0.0.0
+
+gem5 version 22.0 has been slightly delayed, but we a have a very strong release!
+This release has 660 changes from 48 unique contributors.
+While there are not too many big ticket features, the community has done a lot to improve the stablity and add bugfixes to gem5 over this release.
+That said, we have a few cool new features like full system GPU support, a huge number of Arm improvements, and an improved HBM model.
+
+See below for more details!
+
+## New features
+
+- [Arm now models DVM messages for TLBIs and DSBs accurately](https://gem5.atlassian.net/browse/GEM5-1097). This is implemented in the CHI protocol.
+- EL2/EL3 support on by default in ArmSystem
+- HBM controller which supports pseudo channels
+- [Improved Ruby's SimpleNetwork routing](https://gem5.atlassian.net/browse/GEM5-920)
+- Added x86 bare metal workload and better real mode support
+- [Added round-robin arbitration when using multiple prefetchers](https://gem5.atlassian.net/browse/GEM5-1169)
+- [KVM Emulation added for ARM GIGv3](https://gem5.atlassian.net/browse/GEM5-1138)
+- Many improvements to the CHI protocol
+
+## Many RISC-V instructions added
+
+The following RISCV instructions have been added to gem5's RISC-V ISA:
+
+* Zba instructions: add.uw, sh1add, sh1add.uw, sh2add, sh2add.uw, sh3add, sh3add.uw, slli.uw
+* Zbb instructions: andn, orn, xnor, clz, clzw, ctz, ctzw, cpop, cpopw, max, maxu, min, minu, sext.b, sext.h, zext.h, rol, rolw, ror, rori, roriw, rorw, orc.b, rev8
+* Zbc instructions: clmul, clmulh, clmulr
+* Zbs instructions: bclr, bclri, bext, bexti, binv, binvi, bset, bseti
+* Zfh instructions: flh, fsh, fmadd.h, fmsub.h, fnmsub.h, fnmadd.h, fadd.h, fsub.h, fmul.h, fdiv.h, fsqrt.h, fsgnj.h, fsgnjn.h, fsgnjx.h, fmin.h, fmax.h, fcvt.s.h, fcvt.h.s, fcvt.d.h, fcvt.h.d, fcvt.w.h, fcvt.h.w, fcvt.wu.h, fcvt.h.wu
+
+### Improvements to the stdlib automatic resource downloader
+
+The gem5 standard library's downloader has been re-engineered to more efficiently obtain the `resources.json` file.
+It is now cached instead of retrieved on each resource retrieval.
+
+The `resources.json` directory has been moved to a more permament URL at <http://resources.gem5.org/resources.json>.
+
+Tests have also been added to ensure the resources module continues to function correctly.
+
+### gem5 in SystemC support revamped
+
+The gem5 in SystemC has been revamped to accomodate new research needs.
+These changes include stability improvements and bugs fixes.
+The gem5 testing suite has also been expanded to include gem5 in SystemC tests.
+
+### Improved GPU support
+
+Users may now simulate an AMD GPU device in full system mode using the ROCm 4.2 compute stack.
+Until v21.2, gem5 only supported GPU simulation in Syscall-Emulation mode with ROCm 4.0.
+See [`src/gpu-fs/README.md`](https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/gpu-fs/) in gem5-resources and example scripts in [`configs/example/gpufs/`](https://gem5.googlesource.com/public/gem5/+/refs/tags/v22.0.0.0/configs/example/gpufs/) for example scripts which run GPU full system simulations.
+
+A [GPU Ruby random tester has been added](https://gem5-review.googlesource.com/c/public/gem5/+/59272) to help validate the correctness of the CPU and GPU Ruby coherence protocols as part of every kokoro check-in.
+This helps validate the correctness of the protocols before new changes are checked in.
+Currently the tester focuses on the protocols used with the GPU, but the ideas are extensible to other protocols.
+The work is based on "Autonomous Data-Race-Free GPU Testing", IISWC 2019, Tuan Ta, Xianwei Zhang, Anthony Gutierrez, and Bradford M. Beckmann.
+
+### An Arm board has been added to the gem5 Standard Library
+
+Via [this change](https://gem5-review.googlesource.com/c/public/gem5/+/58910), an ARM Board, `ArmBoard`, has been added to the gem5 standard library.
+This allows for an ARM system to be run using the gem5 stdlib components.
+
+An example gem5 configuration script using this board can be found in `configs/example/gem5_library/arm-ubuntu-boot-exit.py`.
+
+### `createAddrRanges` now supports NUMA configurations
+
+When the system is configured for NUMA, it has multiple memory ranges, and each memory range is mapped to a corresponding NUMA node. For this, the change enables `createAddrRanges` to map address ranges to only a given HNFs.
+
+Jira ticker here: https://gem5.atlassian.net/browse/GEM5-1187.
+
+## API (user-facing) changes
+
+### CPU model types are no longer simply the model name, but they are specialized for each ISA
+
+For instance, the `O3CPU` is now the `X86O3CPU` and `ArmO3CPU`, etc.
+This requires a number of changes if you have your own CPU models.
+See https://gem5-review.googlesource.com/c/public/gem5/+/52490 for details.
+
+Additionally, this requires changes in any configuration script which inherits from the old CPU types.
+
+In many cases, if there is only a single ISA compiled the old name will still work.
+However, this is not 100% true.
+
+Finally, `CPU_MODELS` is no longer a parameter in `build_opts/`.
+Now, if you want to compile a CPU model for a particular ISA you will have to add a new file for the CPU model in the `arch/` directory.
+
+### Many changes in the CPU and ISA APIs
+
+If you have any specialized CPU models or any ISAs which are not in the mainline, expect many changes when rebasing on this release.
+
+- No longer use read/setIntReg (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49766)
+- InvalidRegClass has changed (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49745)
+- All of the register classes have changed (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49764/)
+- `initiateSpecialMemCmd` renamed to `initiateMemMgmtCmd` to generalize to other command beyond HTM (e.g., DVM/TLBI)
+- `OperandDesc` class added (e.g., see https://gem5-review.googlesource.com/c/public/gem5/+/49731)
+- Many cases of `TheISA` have been removed
+
+## Bug Fixes
+
+- [Fixed RISC-V call/ret instruction decoding](https://gem5-review.googlesource.com/c/public/gem5/+/58209). The fix adds IsReturn` and `IsCall` flags for RISC-V jump instructions by defining a new `JumpConstructor` in "standard.isa". Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1139.
+- [Fixed x86 Read-Modify-Write behavior in multiple timing cores with classic caches](https://gem5-review.googlesource.com/c/public/gem5/+/55744). Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1105.
+- [The circular buffer for the O3 LSQ has been fixed](https://gem5-review.googlesource.com/c/public/gem5/+/58649). This issue affected running the O3 CPU with large workloaders. Jira Ticket here: https://gem5.atlassian.net/browse/GEM5-1203.
+- [Removed "memory-leak"-like error in RISC-V lr/sc implementation](https://gem5-review.googlesource.com/c/public/gem5/+/55663). Jira issue here: https://gem5.atlassian.net/browse/GEM5-1170.
+- [Resolved issues with Ruby's memtest](https://gem5-review.googlesource.com/c/public/gem5/+/56811). In gem5 v21.2, If the size of the address range was smaller than the maximum number of outstandnig requests allowed downstream, the tester would get stuck trying to find a unique address. This has been resolved.
+
+## Build-related changes
+
+- Variable in `env` in the SConscript files now requires you to use `env['CONF']` to access them. Anywhere that `env['<VARIABLE>']` appeared should noe be `env['CONF']['<VARIABLE>']`
+- Internal build files are now in a per-target `gem5.build` directory
+- All build variable are per-target and there are no longer any shared variables.
+
+## Other changes
+
+- New bootloader is required for Arm VExpress_GEM5_Foundation platform. See https://gem5.atlassian.net/browse/GEM5-1222 for details.
+- The MemCtrl interface has been updated to use more inheritance to make extending it to other memory types (e.g., HBM pseudo channels) easier.
+
# Version 21.2.1.1
**[HOTFIX]** In order to ensure v21 of gem5 remains compatible with future changes, the gem5 stdlib downloader has been updated to obtain the resources.json file from <https://resources.gem5.org/resources.json>.
diff --git a/src/cpu/minor/SConsopts b/SConsopts
similarity index 76%
copy from src/cpu/minor/SConsopts
copy to SConsopts
index 16ff599..bb2de86 100644
--- a/src/cpu/minor/SConsopts
+++ b/SConsopts
@@ -1,6 +1,4 @@
-# -*- mode:python -*-
-
-# Copyright (c) 2012-2014 ARM Limited
+# Copyright (c) 2013, 2015-2020 ARM Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
@@ -12,6 +10,11 @@
# unmodified and in its entirety in all distributions of the software,
# modified or unmodified, in source code or in binary form.
#
+# Copyright (c) 2011 Advanced Micro Devices, Inc.
+# Copyright (c) 2009 The Hewlett-Packard Development Company
+# Copyright (c) 2004-2005 The Regents of The University of Michigan
+# All rights reserved.
+#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
@@ -35,6 +38,16 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import os
+import os.path
+
+from gem5_scons import warning
+
Import('*')
-main.Append(ALL_CPU_MODELS=['MinorCPU'])
+sticky_vars.AddVariables(
+ ('BATCH', 'Use batch pool for build and tests', False),
+ ('BATCH_CMD', 'Batch pool submission command name', 'qdo'),
+ ('M5_BUILD_CACHE', 'Cache built objects in this directory', False),
+ ('USE_EFENCE', 'Link with Electric Fence malloc debugger', False),
+ )
diff --git a/SConstruct b/SConstruct
index ceeb1ba..f1f1c64 100755
--- a/SConstruct
+++ b/SConstruct
@@ -80,17 +80,28 @@
import os
import sys
-from os import mkdir, environ
+from os import mkdir, remove, environ
from os.path import abspath, dirname, expanduser
from os.path import isdir, isfile
from os.path import join, split
+import logging
+logging.basicConfig()
+
# SCons imports
import SCons
import SCons.Node
import SCons.Node.FS
import SCons.Tool
+if getattr(SCons, '__version__', None) in ('3.0.0', '3.0.1'):
+ # Monkey patch a fix which appears in version 3.0.2, since we only
+ # require version 3.0.0
+ def __hash__(self):
+ return hash(self.lstr)
+ import SCons.Subst
+ SCons.Subst.Literal.__hash__ = __hash__
+
########################################################################
#
@@ -98,6 +109,8 @@
#
########################################################################
+linker_options = ('bfd', 'gold', 'lld', 'mold')
+
AddOption('--no-colors', dest='use_colors', action='store_false',
help="Don't add color to abbreviated scons output")
AddOption('--with-cxx-config', action='store_true',
@@ -106,7 +119,10 @@
help='Override which build_opts file to use for defaults')
AddOption('--ignore-style', action='store_true',
help='Disable style checking hooks')
-AddOption('--gold-linker', action='store_true', help='Use the gold linker')
+AddOption('--linker', action='store', default=None, choices=linker_options,
+ help=f'Select which linker to use ({", ".join(linker_options)})')
+AddOption('--gold-linker', action='store_const', const='gold', dest='linker',
+ help='Use the gold linker. Deprecated: Use --linker=gold')
AddOption('--no-compress-debug', action='store_true',
help="Don't compress debug info in build files")
AddOption('--with-lto', action='store_true',
@@ -233,47 +249,6 @@
mkdir(build_root)
main['BUILDROOT'] = build_root
-main.SConsignFile(os.path.join(build_root, "sconsign"))
-
-
-########################################################################
-#
-# Set up global sticky variables... these are common to an entire build
-# tree (not specific to a particular build like X86)
-#
-########################################################################
-
-global_vars_file = os.path.join(build_root, 'variables.global')
-
-global_vars = Variables(global_vars_file, args=ARGUMENTS)
-
-global_vars.AddVariables(
- ('CC', 'C compiler', environ.get('CC', main['CC'])),
- ('CXX', 'C++ compiler', environ.get('CXX', main['CXX'])),
- ('CCFLAGS_EXTRA', 'Extra C and C++ compiler flags', ''),
- ('GEM5PY_CCFLAGS_EXTRA', 'Extra C and C++ gem5py compiler flags', ''),
- ('GEM5PY_LINKFLAGS_EXTRA', 'Extra marshal gem5py flags', ''),
- ('LINKFLAGS_EXTRA', 'Extra linker flags', ''),
- ('PYTHON_CONFIG', 'Python config binary to use',
- [ 'python3-config', 'python-config']
- ),
- ('PROTOC', 'protoc tool', environ.get('PROTOC', 'protoc')),
- ('BATCH', 'Use batch pool for build and tests', False),
- ('BATCH_CMD', 'Batch pool submission command name', 'qdo'),
- ('M5_BUILD_CACHE', 'Cache built objects in this directory', False),
- ('EXTRAS', 'Add extra directories to the compilation', '')
- )
-
-# Update main environment with values from ARGUMENTS & global_vars_file
-global_vars.Update(main)
-Help('''
-Global build variables:
-{help}
-'''.format(help=global_vars.GenerateHelpText(main)), append=True)
-
-# Save sticky variable settings back to current variables file
-global_vars.Save(global_vars_file, main)
-
########################################################################
#
@@ -281,16 +256,8 @@
#
########################################################################
-# Parse EXTRAS variable to build list of all directories where we're
-# look for sources etc. This list is exported as extras_dir_list.
base_dir = Dir('#src').abspath
-if main['EXTRAS']:
- extras_dir_list = makePathListAbsolute(main['EXTRAS'].split(':'))
-else:
- extras_dir_list = []
-
Export('base_dir')
-Export('extras_dir_list')
# the ext directory should be on the #includes path
main.Append(CPPPATH=[Dir('ext')])
@@ -323,151 +290,6 @@
if main['GCC'] + main['CLANG'] > 1:
error('Two compilers enabled at once?')
-# Set up default C++ compiler flags
-if main['GCC'] or main['CLANG']:
- # As gcc and clang share many flags, do the common parts here
- main.Append(CCFLAGS=['-pipe'])
- main.Append(CCFLAGS=['-fno-strict-aliasing'])
-
- # Enable -Wall and -Wextra and then disable the few warnings that
- # we consistently violate
- main.Append(CCFLAGS=['-Wall', '-Wundef', '-Wextra',
- '-Wno-sign-compare', '-Wno-unused-parameter'])
-
- # We always compile using C++17
- main.Append(CXXFLAGS=['-std=c++17'])
-
- if sys.platform.startswith('freebsd'):
- main.Append(CCFLAGS=['-I/usr/local/include'])
- main.Append(CXXFLAGS=['-I/usr/local/include'])
- # On FreeBSD we need libthr.
- main.Append(LIBS=['thr'])
-
- with gem5_scons.Configure(main) as conf:
- conf.CheckLinkFlag('-Wl,--as-needed')
- if GetOption('gold_linker'):
- main.Append(LINKFLAGS='-fuse-ld=gold')
-
-else:
- error('\n'.join((
- "Don't know what compiler options to use for your compiler.",
- "compiler: " + main['CXX'],
- "version: " + CXX_version.replace('\n', '<nl>') if
- CXX_version else 'COMMAND NOT FOUND!',
- "If you're trying to use a compiler other than GCC",
- "or clang, there appears to be something wrong with your",
- "environment.",
- "",
- "If you are trying to use a compiler other than those listed",
- "above you will need to ease fix SConstruct and ",
- "src/SConscript to support that compiler.")))
-
-if main['GCC']:
- if compareVersions(main['CXXVERSION'], "7") < 0:
- error('gcc version 7 or newer required.\n'
- 'Installed version:', main['CXXVERSION'])
-
- with gem5_scons.Configure(main) as conf:
- # This warning has a false positive in the systemc code in g++ 11.1.
- conf.CheckCxxFlag('-Wno-free-nonheap-object')
-
- # Add the appropriate Link-Time Optimization (LTO) flags if `--with-lto` is
- # set.
- if GetOption('with_lto'):
- # g++ uses "make" to parallelize LTO. The program can be overriden with
- # the environment variable "MAKE", but we currently make no attempt to
- # plumb that variable through.
- parallelism = ''
- if main.Detect('make'):
- parallelism = '=%d' % GetOption('num_jobs')
- else:
- warning('"make" not found, link time optimization will be '
- 'single threaded.')
-
- for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
- # Use the same amount of jobs for LTO as we are running scons with.
- main[var] = ['-flto%s' % parallelism]
-
- main.Append(TCMALLOC_CCFLAGS=['-fno-builtin-malloc', '-fno-builtin-calloc',
- '-fno-builtin-realloc', '-fno-builtin-free'])
-
-elif main['CLANG']:
- if compareVersions(main['CXXVERSION'], "6") < 0:
- error('clang version 6 or newer required.\n'
- 'Installed version:', main['CXXVERSION'])
-
- # Set the Link-Time Optimization (LTO) flags if enabled.
- if GetOption('with_lto'):
- for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
- main[var] = ['-flto']
-
- # clang has a few additional warnings that we disable.
- with gem5_scons.Configure(main) as conf:
- conf.CheckCxxFlag('-Wno-c99-designator')
- conf.CheckCxxFlag('-Wno-defaulted-function-deleted')
-
- main.Append(TCMALLOC_CCFLAGS=['-fno-builtin'])
-
- # On Mac OS X/Darwin we need to also use libc++ (part of XCode) as
- # opposed to libstdc++, as the later is dated.
- if sys.platform == "darwin":
- main.Append(CXXFLAGS=['-stdlib=libc++'])
- main.Append(LIBS=['c++'])
-
-# Add sanitizers flags
-sanitizers=[]
-if GetOption('with_ubsan'):
- sanitizers.append('undefined')
-if GetOption('with_asan'):
- # Available for gcc >= 5 or llvm >= 3.1 both a requirement
- # by the build system
- sanitizers.append('address')
- suppressions_file = Dir('util').File('lsan-suppressions').get_abspath()
- suppressions_opt = 'suppressions=%s' % suppressions_file
- suppressions_opts = ':'.join([suppressions_opt, 'print_suppressions=0'])
- main['ENV']['LSAN_OPTIONS'] = suppressions_opts
- print()
- warning('To suppress false positive leaks, set the LSAN_OPTIONS '
- 'environment variable to "%s" when running gem5' %
- suppressions_opts)
- warning('LSAN_OPTIONS=%s' % suppressions_opts)
- print()
-if sanitizers:
- sanitizers = ','.join(sanitizers)
- if main['GCC'] or main['CLANG']:
- main.Append(CCFLAGS=['-fsanitize=%s' % sanitizers,
- '-fno-omit-frame-pointer'],
- LINKFLAGS='-fsanitize=%s' % sanitizers)
- else:
- warning("Don't know how to enable %s sanitizer(s) for your "
- "compiler." % sanitizers)
-
-# Do this after we save setting back, or else we'll tack on an
-# extra 'qdo' every time we run scons.
-if main['BATCH']:
- main['CC'] = main['BATCH_CMD'] + ' ' + main['CC']
- main['CXX'] = main['BATCH_CMD'] + ' ' + main['CXX']
- main['AS'] = main['BATCH_CMD'] + ' ' + main['AS']
- main['AR'] = main['BATCH_CMD'] + ' ' + main['AR']
- main['RANLIB'] = main['BATCH_CMD'] + ' ' + main['RANLIB']
-
-if sys.platform == 'cygwin':
- # cygwin has some header file issues...
- main.Append(CCFLAGS=["-Wno-uninitialized"])
-
-
-# Cache build files in the supplied directory.
-if main['M5_BUILD_CACHE']:
- print('Using build cache located at', main['M5_BUILD_CACHE'])
- CacheDir(main['M5_BUILD_CACHE'])
-
-if not GetOption('no_compress_debug'):
- with gem5_scons.Configure(main) as conf:
- if not conf.CheckCxxFlag('-gz'):
- warning("Can't enable object file debug section compression")
- if not conf.CheckLinkFlag('-gz'):
- warning("Can't enable executable debug section compression")
-
########################################################################
#
@@ -532,112 +354,6 @@
warning('Embedded python library too new. '
f'Python 3 expected, found {ver_string}.')
-if main['USE_PYTHON']:
- config_embedded_python(main)
- gem5py_env = main.Clone()
-else:
- gem5py_env = main.Clone()
- config_embedded_python(gem5py_env)
-
-# Bare minimum environment that only includes python
-gem5py_env.Append(CCFLAGS=['${GEM5PY_CCFLAGS_EXTRA}'])
-gem5py_env.Append(LINKFLAGS=['${GEM5PY_LINKFLAGS_EXTRA}'])
-
-if GetOption('gprof') and GetOption('pprof'):
- error('Only one type of profiling should be enabled at a time')
-if GetOption('gprof'):
- main.Append(CCFLAGS=['-g', '-pg'], LINKFLAGS=['-pg'])
-if GetOption('pprof'):
- main.Append(CCFLAGS=['-g'],
- LINKFLAGS=['-Wl,--no-as-needed', '-lprofiler', '-Wl,--as-needed'])
-
-main['HAVE_PKG_CONFIG'] = main.Detect('pkg-config')
-
-with gem5_scons.Configure(main) as conf:
- # On Solaris you need to use libsocket for socket ops
- if not conf.CheckLibWithHeader(
- [None, 'socket'], 'sys/socket.h', 'C++', 'accept(0,0,0);'):
- error("Can't find library with socket calls (e.g. accept()).")
-
- if not conf.CheckLibWithHeader('z', 'zlib.h', 'C++','zlibVersion();'):
- error('Did not find needed zlib compression library '
- 'and/or zlib.h header file.\n'
- 'Please install zlib and try again.')
-
-if not GetOption('without_tcmalloc'):
- with gem5_scons.Configure(main) as conf:
- if conf.CheckLib('tcmalloc'):
- conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
- elif conf.CheckLib('tcmalloc_minimal'):
- conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
- else:
- warning("You can get a 12% performance improvement by "
- "installing tcmalloc (libgoogle-perftools-dev package "
- "on Ubuntu or RedHat).")
-
-
-########################################################################
-#
-# Read and process SConsopts files. These can add new settings which
-# affect each variant directory independently.
-#
-########################################################################
-
-# Register a callback which is called after all SConsopts files have been read.
-after_sconsopts_callbacks = []
-def AfterSConsopts(cb):
- after_sconsopts_callbacks.append(cb)
-Export('AfterSConsopts')
-
-# Sticky variables get saved in the variables file so they persist from
-# one invocation to the next (unless overridden, in which case the new
-# value becomes sticky).
-sticky_vars = Variables(args=ARGUMENTS)
-Export('sticky_vars')
-
-# Sticky variables that should be exported to #defines in config/*.hh
-# (see src/SConscript).
-export_vars = []
-Export('export_vars')
-
-# Walk the tree and execute all SConsopts scripts that wil add to the
-# above variables
-if GetOption('verbose'):
- print("Reading SConsopts")
-for bdir in [ base_dir ] + extras_dir_list:
- if not isdir(bdir):
- error("Directory '%s' does not exist." % bdir)
- for root, dirs, files in os.walk(bdir):
- if 'SConsopts' in files:
- if GetOption('verbose'):
- print("Reading", os.path.join(root, 'SConsopts'))
- SConscript(os.path.join(root, 'SConsopts'))
-
-# Call any callbacks which the SConsopts files registered.
-for cb in after_sconsopts_callbacks:
- cb()
-
-# Add any generic sticky variables here.
-sticky_vars.Add(BoolVariable('USE_EFENCE',
- 'Link with Electric Fence malloc debugger', False))
-
-
-########################################################################
-#
-# Find and process all the SConscript files in ext. These are shared by
-# all variants in a build root.
-#
-########################################################################
-
-ext_dir = Dir('#ext').abspath
-ext_build_dirs = []
-for root, dirs, files in os.walk(ext_dir):
- if 'SConscript' in files:
- build_dir = os.path.relpath(root, ext_dir)
- ext_build_dirs.append(build_dir)
- main.SConscript(os.path.join(root, 'SConscript'),
- variant_dir=os.path.join(build_root, build_dir))
-
########################################################################
#
@@ -646,38 +362,260 @@
########################################################################
for variant_path in variant_paths:
- if not GetOption('silent'):
- print("Building in", variant_path)
-
# Make a copy of the build-root environment to use for this config.
env = main.Clone()
env['BUILDDIR'] = variant_path
+ gem5_build = os.path.join(build_root, variant_path, 'gem5.build')
+ env['GEM5BUILD'] = gem5_build
+ Execute(Mkdir(gem5_build))
+
+ env.SConsignFile(os.path.join(gem5_build, 'sconsign'))
+
+ # Set up default C++ compiler flags
+ if env['GCC'] or env['CLANG']:
+ # As gcc and clang share many flags, do the common parts here
+ env.Append(CCFLAGS=['-pipe'])
+ env.Append(CCFLAGS=['-fno-strict-aliasing'])
+
+ # Enable -Wall and -Wextra and then disable the few warnings that
+ # we consistently violate
+ env.Append(CCFLAGS=['-Wall', '-Wundef', '-Wextra',
+ '-Wno-sign-compare', '-Wno-unused-parameter'])
+
+ # We always compile using C++17
+ env.Append(CXXFLAGS=['-std=c++17'])
+
+ if sys.platform.startswith('freebsd'):
+ env.Append(CCFLAGS=['-I/usr/local/include'])
+ env.Append(CXXFLAGS=['-I/usr/local/include'])
+ # On FreeBSD we need libthr.
+ env.Append(LIBS=['thr'])
+
+ with gem5_scons.Configure(env) as conf:
+ conf.CheckLinkFlag('-Wl,--as-needed')
+
+ linker = GetOption('linker')
+ if linker:
+ with gem5_scons.Configure(env) as conf:
+ if not conf.CheckLinkFlag(f'-fuse-ld={linker}'):
+ # check mold support for gcc older than 12.1.0
+ if linker == 'mold' and \
+ (env['GCC'] and \
+ compareVersions(env['CXXVERSION'],
+ "12.1.0") < 0) and \
+ ((isdir('/usr/libexec/mold') and \
+ conf.CheckLinkFlag('-B/usr/libexec/mold')) or \
+ (isdir('/usr/local/libexec/mold') and \
+ conf.CheckLinkFlag('-B/usr/local/libexec/mold'))):
+ pass # support mold
+ else:
+ error(f'Linker "{linker}" is not supported')
+ if linker == 'gold' and not GetOption('with_lto'):
+ # Tell the gold linker to use threads. The gold linker
+ # segfaults if both threads and LTO are enabled.
+ conf.CheckLinkFlag('-Wl,--threads')
+ conf.CheckLinkFlag(
+ '-Wl,--thread-count=%d' % GetOption('num_jobs'))
+
+ else:
+ error('\n'.join((
+ "Don't know what compiler options to use for your compiler.",
+ "compiler: " + env['CXX'],
+ "version: " + CXX_version.replace('\n', '<nl>') if
+ CXX_version else 'COMMAND NOT FOUND!',
+ "If you're trying to use a compiler other than GCC",
+ "or clang, there appears to be something wrong with your",
+ "environment.",
+ "",
+ "If you are trying to use a compiler other than those listed",
+ "above you will need to ease fix SConstruct and ",
+ "src/SConscript to support that compiler.")))
+
+ if env['GCC']:
+ if compareVersions(env['CXXVERSION'], "7") < 0:
+ error('gcc version 7 or newer required.\n'
+ 'Installed version:', env['CXXVERSION'])
+
+ with gem5_scons.Configure(env) as conf:
+ # This warning has a false positive in the systemc in g++ 11.1.
+ conf.CheckCxxFlag('-Wno-free-nonheap-object')
+
+ # Add the appropriate Link-Time Optimization (LTO) flags if
+ # `--with-lto` is set.
+ if GetOption('with_lto'):
+ # g++ uses "make" to parallelize LTO. The program can be overriden
+ # with the environment variable "MAKE", but we currently make no
+ # attempt to plumb that variable through.
+ parallelism = ''
+ if env.Detect('make'):
+ parallelism = '=%d' % GetOption('num_jobs')
+ else:
+ warning('"make" not found, link time optimization will be '
+ 'single threaded.')
+
+ for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
+ # Use the same amount of jobs for LTO as scons.
+ env[var] = ['-flto%s' % parallelism]
+
+ env.Append(TCMALLOC_CCFLAGS=[
+ '-fno-builtin-malloc', '-fno-builtin-calloc',
+ '-fno-builtin-realloc', '-fno-builtin-free'])
+
+ elif env['CLANG']:
+ if compareVersions(env['CXXVERSION'], "6") < 0:
+ error('clang version 6 or newer required.\n'
+ 'Installed version:', env['CXXVERSION'])
+
+ # Set the Link-Time Optimization (LTO) flags if enabled.
+ if GetOption('with_lto'):
+ for var in 'LTO_CCFLAGS', 'LTO_LINKFLAGS':
+ env[var] = ['-flto']
+
+ # clang has a few additional warnings that we disable.
+ with gem5_scons.Configure(env) as conf:
+ conf.CheckCxxFlag('-Wno-c99-designator')
+ conf.CheckCxxFlag('-Wno-defaulted-function-deleted')
+
+ env.Append(TCMALLOC_CCFLAGS=['-fno-builtin'])
+
+ # On Mac OS X/Darwin we need to also use libc++ (part of XCode) as
+ # opposed to libstdc++, as the later is dated.
+ if sys.platform == "darwin":
+ env.Append(CXXFLAGS=['-stdlib=libc++'])
+ env.Append(LIBS=['c++'])
+
+ # Add sanitizers flags
+ sanitizers=[]
+ if GetOption('with_ubsan'):
+ sanitizers.append('undefined')
+ if GetOption('with_asan'):
+ # Available for gcc >= 5 or llvm >= 3.1 both a requirement
+ # by the build system
+ sanitizers.append('address')
+ suppressions_file = Dir('util').File('lsan-suppressions').get_abspath()
+ suppressions_opt = 'suppressions=%s' % suppressions_file
+ suppressions_opts = ':'.join([suppressions_opt,
+ 'print_suppressions=0'])
+ env['ENV']['LSAN_OPTIONS'] = suppressions_opts
+ print()
+ warning('To suppress false positive leaks, set the LSAN_OPTIONS '
+ 'environment variable to "%s" when running gem5' %
+ suppressions_opts)
+ warning('LSAN_OPTIONS=%s' % suppressions_opts)
+ print()
+ if sanitizers:
+ sanitizers = ','.join(sanitizers)
+ if env['GCC'] or env['CLANG']:
+ env.Append(CCFLAGS=['-fsanitize=%s' % sanitizers,
+ '-fno-omit-frame-pointer'],
+ LINKFLAGS='-fsanitize=%s' % sanitizers)
+ else:
+ warning("Don't know how to enable %s sanitizer(s) for your "
+ "compiler." % sanitizers)
+
+ if sys.platform == 'cygwin':
+ # cygwin has some header file issues...
+ env.Append(CCFLAGS=["-Wno-uninitialized"])
+
+
+ if not GetOption('no_compress_debug'):
+ with gem5_scons.Configure(env) as conf:
+ if not conf.CheckCxxFlag('-gz'):
+ warning("Can't enable object file debug section compression")
+ if not conf.CheckLinkFlag('-gz'):
+ warning("Can't enable executable debug section compression")
+
+ if env['USE_PYTHON']:
+ config_embedded_python(env)
+ gem5py_env = env.Clone()
+ else:
+ gem5py_env = env.Clone()
+ config_embedded_python(gem5py_env)
+
+ # Bare minimum environment that only includes python
+ gem5py_env.Append(CCFLAGS=['${GEM5PY_CCFLAGS_EXTRA}'])
+ gem5py_env.Append(LINKFLAGS=['${GEM5PY_LINKFLAGS_EXTRA}'])
+
+ if GetOption('gprof') and GetOption('pprof'):
+ error('Only one type of profiling should be enabled at a time')
+ if GetOption('gprof'):
+ env.Append(CCFLAGS=['-g', '-pg'], LINKFLAGS=['-pg'])
+ if GetOption('pprof'):
+ env.Append(CCFLAGS=['-g'],
+ LINKFLAGS=['-Wl,--no-as-needed', '-lprofiler',
+ '-Wl,--as-needed'])
+
+ env['HAVE_PKG_CONFIG'] = env.Detect('pkg-config')
+
+ with gem5_scons.Configure(env) as conf:
+ # On Solaris you need to use libsocket for socket ops
+ if not conf.CheckLibWithHeader(
+ [None, 'socket'], 'sys/socket.h', 'C++', 'accept(0,0,0);'):
+ error("Can't find library with socket calls (e.g. accept()).")
+
+ if not conf.CheckLibWithHeader('z', 'zlib.h', 'C++','zlibVersion();'):
+ error('Did not find needed zlib compression library '
+ 'and/or zlib.h header file.\n'
+ 'Please install zlib and try again.')
+
+ if not GetOption('without_tcmalloc'):
+ with gem5_scons.Configure(env) as conf:
+ if conf.CheckLib('tcmalloc'):
+ conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
+ elif conf.CheckLib('tcmalloc_minimal'):
+ conf.env.Append(CCFLAGS=conf.env['TCMALLOC_CCFLAGS'])
+ else:
+ warning("You can get a 12% performance improvement by "
+ "installing tcmalloc (libgoogle-perftools-dev package "
+ "on Ubuntu or RedHat).")
+
+ if not GetOption('silent'):
+ print("Building in", variant_path)
+
# variant_dir is the tail component of build path, and is used to
# determine the build parameters (e.g., 'X86')
(build_root, variant_dir) = os.path.split(variant_path)
+ ####################################################################
+ #
+ # Read and process SConsopts files. These can add new settings which
+ # affect each variant directory independently.
+ #
+ ####################################################################
+
+ # Register a callback to call after all SConsopts files have been read.
+ after_sconsopts_callbacks = []
+ def AfterSConsopts(cb):
+ after_sconsopts_callbacks.append(cb)
+ Export('AfterSConsopts')
+
+ # Sticky variables get saved in the variables file so they persist from
+ # one invocation to the next (unless overridden, in which case the new
+ # value becomes sticky).
+ sticky_vars = Variables(args=ARGUMENTS)
+ Export('sticky_vars')
+
+ # EXTRAS is special since it affects what SConsopts need to be read.
+ sticky_vars.Add(('EXTRAS', 'Add extra directories to the compilation', ''))
+
# Set env variables according to the build directory config.
sticky_vars.files = []
# Variables for $BUILD_ROOT/$VARIANT_DIR are stored in
- # $BUILD_ROOT/variables/$VARIANT_DIR so you can nuke
- # $BUILD_ROOT/$VARIANT_DIR without losing your variables settings.
- current_vars_file = os.path.join(build_root, 'variables', variant_dir)
- if isfile(current_vars_file):
- sticky_vars.files.append(current_vars_file)
+ # $BUILD_ROOT/$VARIANT_DIR/gem5.build/variables
+
+ gem5_build_vars = os.path.join(gem5_build, 'variables')
+ build_root_vars = os.path.join(build_root, 'variables', variant_dir)
+ current_vars_files = [gem5_build_vars, build_root_vars]
+ existing_vars_files = list(filter(isfile, current_vars_files))
+ if existing_vars_files:
+ sticky_vars.files.extend(existing_vars_files)
if not GetOption('silent'):
- print("Using saved variables file %s" % current_vars_file)
- elif variant_dir in ext_build_dirs:
- # Things in ext are built without a variant directory.
- continue
+ print('Using saved variables file(s) %s' %
+ ', '.join(existing_vars_files))
else:
# Variant specific variables file doesn't exist.
- # Make sure the directory is there so we can create the file later.
- opt_dir = dirname(current_vars_file)
- if not isdir(opt_dir):
- mkdir(opt_dir)
-
# Get default build variables from source tree. Variables are
# normally determined by name of $VARIANT_DIR, but can be
# overridden by '--default=' arg on command line.
@@ -685,23 +623,64 @@
opts_dir = Dir('#build_opts').abspath
if default:
default_vars_files = [
- os.path.join(build_root, 'variables', default),
+ gem5_build_vars,
+ build_root_vars,
os.path.join(opts_dir, default)
]
else:
default_vars_files = [os.path.join(opts_dir, variant_dir)]
- existing_files = list(filter(isfile, default_vars_files))
- if existing_files:
- default_vars_file = existing_files[0]
+ existing_default_files = list(filter(isfile, default_vars_files))
+ if existing_default_files:
+ default_vars_file = existing_default_files[0]
sticky_vars.files.append(default_vars_file)
- print("Variables file %s not found,\n using defaults in %s"
- % (current_vars_file, default_vars_file))
+ print("Variables file(s) %s not found,\n using defaults in %s" %
+ (' or '.join(current_vars_files), default_vars_file))
else:
- error("Cannot find variables file %s or default file(s) %s"
- % (current_vars_file, ' or '.join(default_vars_files)))
+ error("Cannot find variables file(s) %s or default file(s) %s" %
+ (' or '.join(current_vars_files),
+ ' or '.join(default_vars_files)))
Exit(1)
- # Apply current variable settings to env
+ # Apply current settings for EXTRAS to env.
+ sticky_vars.Update(env)
+
+ # Parse EXTRAS variable to build list of all directories where we're
+ # look for sources etc. This list is exported as extras_dir_list.
+ if env['EXTRAS']:
+ extras_dir_list = makePathListAbsolute(env['EXTRAS'].split(':'))
+ else:
+ extras_dir_list = []
+
+ Export('extras_dir_list')
+
+ # Variables which were determined with Configure.
+ env['CONF'] = {}
+
+ # Walk the tree and execute all SConsopts scripts that wil add to the
+ # above variables
+ if GetOption('verbose'):
+ print("Reading SConsopts")
+
+ def trySConsopts(dir):
+ sconsopts_path = os.path.join(dir, 'SConsopts')
+ if not isfile(sconsopts_path):
+ return
+ if GetOption('verbose'):
+ print("Reading", sconsopts_path)
+ SConscript(sconsopts_path, exports={'main': env})
+
+ trySConsopts(Dir('#').abspath)
+ for bdir in [ base_dir ] + extras_dir_list:
+ if not isdir(bdir):
+ error("Directory '%s' does not exist." % bdir)
+ for root, dirs, files in os.walk(bdir):
+ trySConsopts(root)
+
+ # Call any callbacks which the SConsopts files registered.
+ for cb in after_sconsopts_callbacks:
+ cb()
+
+ # Update env for new variables added by the SConsopts.
sticky_vars.Update(env)
Help('''
@@ -710,21 +689,45 @@
'''.format(dir=variant_dir, help=sticky_vars.GenerateHelpText(env)),
append=True)
- # Process variable settings.
- if env['USE_EFENCE']:
- env.Append(LIBS=['efence'])
+ # If the old vars file exists, delete it to avoid confusion/stale values.
+ if isfile(build_root_vars):
+ warning(f'Deleting old variant variables file "{build_root_vars}"')
+ remove(build_root_vars)
+ # Save sticky variables back to the gem5.build variant variables file.
+ sticky_vars.Save(gem5_build_vars, env)
- if env['KVM_ISA'] != env['TARGET_ISA']:
- env['USE_KVM'] = False
+ # Pull all the sticky variables into the CONF dict.
+ env['CONF'].update({key: env[key] for key in sticky_vars.keys()})
- # Save sticky variable settings back to current variables file
- sticky_vars.Save(current_vars_file, env)
+ # Do this after we save setting back, or else we'll tack on an
+ # extra 'qdo' every time we run scons.
+ if env['CONF']['BATCH']:
+ env['CC'] = env['CONF']['BATCH_CMD'] + ' ' + env['CC']
+ env['CXX'] = env['CONF']['BATCH_CMD'] + ' ' + env['CXX']
+ env['AS'] = env['CONF']['BATCH_CMD'] + ' ' + env['AS']
+ env['AR'] = env['CONF']['BATCH_CMD'] + ' ' + env['AR']
+ env['RANLIB'] = env['CONF']['BATCH_CMD'] + ' ' + env['RANLIB']
+
+ # Cache build files in the supplied directory.
+ if env['CONF']['M5_BUILD_CACHE']:
+ print('Using build cache located at', env['CONF']['M5_BUILD_CACHE'])
+ CacheDir(env['CONF']['M5_BUILD_CACHE'])
+
env.Append(CCFLAGS='$CCFLAGS_EXTRA')
env.Append(LINKFLAGS='$LINKFLAGS_EXTRA')
exports=['env', 'gem5py_env']
+ ext_dir = Dir('#ext').abspath
+ variant_ext = os.path.join(variant_path, 'ext')
+ for root, dirs, files in os.walk(ext_dir):
+ if 'SConscript' in files:
+ build_dir = os.path.relpath(root, ext_dir)
+ SConscript(os.path.join(root, 'SConscript'),
+ variant_dir=os.path.join(variant_ext, build_dir),
+ exports=exports)
+
# The src/SConscript file sets up the build rules in 'env' according
# to the configured variables. It returns a list of environments,
# one for each variant build (debug, opt, etc.)
diff --git a/build_opts/ARM b/build_opts/ARM
index e4cb9a5..5b7da10 100644
--- a/build_opts/ARM
+++ b/build_opts/ARM
@@ -1,3 +1,2 @@
TARGET_ISA = 'arm'
-CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU,MinorCPU'
PROTOCOL = 'CHI'
diff --git a/build_opts/ARM_MESI_Three_Level b/build_opts/ARM_MESI_Three_Level
index 29a429c..2ca31b6 100644
--- a/build_opts/ARM_MESI_Three_Level
+++ b/build_opts/ARM_MESI_Three_Level
@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MESI_Three_Level'
diff --git a/build_opts/ARM_MESI_Three_Level_HTM b/build_opts/ARM_MESI_Three_Level_HTM
index fd7c164..703398d 100644
--- a/build_opts/ARM_MESI_Three_Level_HTM
+++ b/build_opts/ARM_MESI_Three_Level_HTM
@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MESI_Three_Level_HTM'
diff --git a/build_opts/ARM_MOESI_hammer b/build_opts/ARM_MOESI_hammer
index 2ba8ce8..bd5c63f 100644
--- a/build_opts/ARM_MOESI_hammer
+++ b/build_opts/ARM_MOESI_hammer
@@ -2,5 +2,4 @@
# All rights reserved.
TARGET_ISA = 'arm'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU'
PROTOCOL = 'MOESI_hammer'
diff --git a/build_opts/GCN3_X86 b/build_opts/GCN3_X86
index 21e3cf0..b396908 100644
--- a/build_opts/GCN3_X86
+++ b/build_opts/GCN3_X86
@@ -2,4 +2,3 @@
TARGET_ISA = 'x86'
TARGET_GPU_ISA = 'gcn3'
BUILD_GPU = True
-CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'
diff --git a/build_opts/Garnet_standalone b/build_opts/Garnet_standalone
index f749d54..fd730c3 100644
--- a/build_opts/Garnet_standalone
+++ b/build_opts/Garnet_standalone
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL = 'Garnet_standalone'
diff --git a/build_opts/MIPS b/build_opts/MIPS
index ecb2b09..26cb23c 100644
--- a/build_opts/MIPS
+++ b/build_opts/MIPS
@@ -1,3 +1,2 @@
TARGET_ISA = 'mips'
-CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'
diff --git a/build_opts/NULL b/build_opts/NULL
index 1242fa9..b749729 100644
--- a/build_opts/NULL
+++ b/build_opts/NULL
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL='MI_example'
diff --git a/build_opts/NULL_MESI_Two_Level b/build_opts/NULL_MESI_Two_Level
index db05046..09147b2 100644
--- a/build_opts/NULL_MESI_Two_Level
+++ b/build_opts/NULL_MESI_Two_Level
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL = 'MESI_Two_Level'
diff --git a/build_opts/NULL_MOESI_CMP_directory b/build_opts/NULL_MOESI_CMP_directory
index 7136d0b..466a268 100644
--- a/build_opts/NULL_MOESI_CMP_directory
+++ b/build_opts/NULL_MOESI_CMP_directory
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL='MOESI_CMP_directory'
diff --git a/build_opts/NULL_MOESI_CMP_token b/build_opts/NULL_MOESI_CMP_token
index 42fff75..0cd0305 100644
--- a/build_opts/NULL_MOESI_CMP_token
+++ b/build_opts/NULL_MOESI_CMP_token
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL='MOESI_CMP_token'
diff --git a/build_opts/NULL_MOESI_hammer b/build_opts/NULL_MOESI_hammer
index ff4c22c..39ebcae 100644
--- a/build_opts/NULL_MOESI_hammer
+++ b/build_opts/NULL_MOESI_hammer
@@ -1,3 +1,2 @@
TARGET_ISA = 'null'
-CPU_MODELS = ''
PROTOCOL='MOESI_hammer'
diff --git a/build_opts/POWER b/build_opts/POWER
index 672046c..35772a4 100644
--- a/build_opts/POWER
+++ b/build_opts/POWER
@@ -1,3 +1,2 @@
TARGET_ISA = 'power'
-CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'
diff --git a/build_opts/RISCV b/build_opts/RISCV
index 38abd92..0bd069d 100644
--- a/build_opts/RISCV
+++ b/build_opts/RISCV
@@ -1,3 +1,2 @@
TARGET_ISA = 'riscv'
-CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,MinorCPU,O3CPU'
PROTOCOL = 'MI_example'
diff --git a/build_opts/SPARC b/build_opts/SPARC
index 63ec7cb..98acfe2 100644
--- a/build_opts/SPARC
+++ b/build_opts/SPARC
@@ -1,3 +1,2 @@
TARGET_ISA = 'sparc'
-CPU_MODELS = 'AtomicSimpleCPU,TimingSimpleCPU,O3CPU'
PROTOCOL = 'MI_example'
diff --git a/build_opts/VEGA_X86 b/build_opts/VEGA_X86
index 796b556..11e8232 100644
--- a/build_opts/VEGA_X86
+++ b/build_opts/VEGA_X86
@@ -2,4 +2,3 @@
TARGET_ISA = 'x86'
TARGET_GPU_ISA = 'vega'
BUILD_GPU = True
-CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'
diff --git a/build_opts/X86 b/build_opts/X86
index 27c1b58..72b200a 100644
--- a/build_opts/X86
+++ b/build_opts/X86
@@ -1,4 +1,3 @@
TARGET_ISA = 'x86'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MESI_Two_Level'
-NUMBER_BITS_PER_SET = '128'
\ No newline at end of file
+NUMBER_BITS_PER_SET = '128'
diff --git a/build_opts/X86_MESI_Two_Level b/build_opts/X86_MESI_Two_Level
index eba850b..72b200a 100644
--- a/build_opts/X86_MESI_Two_Level
+++ b/build_opts/X86_MESI_Two_Level
@@ -1,4 +1,3 @@
TARGET_ISA = 'x86'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MESI_Two_Level'
NUMBER_BITS_PER_SET = '128'
diff --git a/build_opts/X86_MI_example b/build_opts/X86_MI_example
index 60d1645..483cf04 100644
--- a/build_opts/X86_MI_example
+++ b/build_opts/X86_MI_example
@@ -1,3 +1,2 @@
TARGET_ISA = 'x86'
-CPU_MODELS = 'TimingSimpleCPU,O3CPU,AtomicSimpleCPU'
PROTOCOL = 'MI_example'
diff --git a/build_opts/X86_MOESI_AMD_Base b/build_opts/X86_MOESI_AMD_Base
index e85f36d..261bedb 100644
--- a/build_opts/X86_MOESI_AMD_Base
+++ b/build_opts/X86_MOESI_AMD_Base
@@ -1,3 +1,2 @@
PROTOCOL = 'MOESI_AMD_Base'
TARGET_ISA = 'x86'
-CPU_MODELS = 'AtomicSimpleCPU,O3CPU,TimingSimpleCPU'
\ No newline at end of file
diff --git a/build_tools/cxx_config_cc.py b/build_tools/cxx_config_cc.py
new file mode 100644
index 0000000..c4a2d89
--- /dev/null
+++ b/build_tools/cxx_config_cc.py
@@ -0,0 +1,299 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
+# Copyright 2021 Google, Inc.
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import argparse
+import importlib
+import os.path
+import sys
+
+import importer
+
+from code_formatter import code_formatter
+
+parser = argparse.ArgumentParser()
+parser.add_argument('modpath', help='module the simobject belongs to')
+parser.add_argument('cxx_config_cc', help='cxx config cc file to generate')
+
+args = parser.parse_args()
+
+basename = os.path.basename(args.cxx_config_cc)
+sim_object_name = os.path.splitext(basename)[0]
+
+importer.install()
+module = importlib.import_module(args.modpath)
+sim_object = getattr(module, sim_object_name)
+
+from m5.params import isSimObjectClass
+import m5.params
+
+code = code_formatter()
+
+entry_class = 'CxxConfigDirectoryEntry_%s' % sim_object_name
+param_class = '%sCxxConfigParams' % sim_object_name
+
+def cxx_bool(b):
+ return 'true' if b else 'false'
+
+code('#include "params/%s.hh"' % sim_object_name)
+
+for param in sim_object._params.values():
+ if isSimObjectClass(param.ptype):
+ code('#include "%s"' % param.ptype._value_dict['cxx_header'])
+ code('#include "params/%s.hh"' % param.ptype.__name__)
+ else:
+ param.ptype.cxx_ini_predecls(code)
+
+code('''#include "${{sim_object._value_dict['cxx_header']}}"
+#include "base/str.hh"
+#include "cxx_config/${sim_object_name}.hh"
+
+namespace gem5
+{
+
+${param_class}::DirectoryEntry::DirectoryEntry()
+{
+''')
+code.indent()
+for param in sim_object._params.values():
+ is_vector = isinstance(param, m5.params.VectorParamDesc)
+ is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
+
+ code('parameters["%s"] = new ParamDesc("%s", %s, %s);' %
+ (param.name, param.name, cxx_bool(is_vector),
+ cxx_bool(is_simobj)));
+
+for port in sim_object._ports.values():
+ is_vector = isinstance(port, m5.params.VectorPort)
+ is_requestor = port.role == 'GEM5 REQUESTOR'
+
+ code('ports["%s"] = new PortDesc("%s", %s, %s);' %
+ (port.name, port.name, cxx_bool(is_vector),
+ cxx_bool(is_requestor)))
+
+code.dedent()
+
+code('''}
+
+bool
+${param_class}::setSimObject(const std::string &name, SimObject *simObject)
+{
+ bool ret = true;
+ if (false) {
+''')
+
+code.indent()
+for param in sim_object._params.values():
+ is_vector = isinstance(param, m5.params.VectorParamDesc)
+ is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
+
+ if is_simobj and not is_vector:
+ code('} else if (name == "${{param.name}}") {')
+ code.indent()
+ code('this->${{param.name}} = '
+ 'dynamic_cast<${{param.ptype.cxx_type}}>(simObject);')
+ code('if (simObject && !this->${{param.name}})')
+ code(' ret = false;')
+ code.dedent()
+code.dedent()
+
+code('''
+ } else {
+ ret = false;
+ }
+
+ return ret;
+}
+
+bool
+${param_class}::setSimObjectVector(const std::string &name,
+ const std::vector<SimObject *> &simObjects)
+{
+ bool ret = true;
+
+ if (false) {
+''')
+
+code.indent()
+for param in sim_object._params.values():
+ is_vector = isinstance(param, m5.params.VectorParamDesc)
+ is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
+
+ if is_simobj and is_vector:
+ code('} else if (name == "${{param.name}}") {')
+ code.indent()
+ code('this->${{param.name}}.clear();')
+ code('for (auto i = simObjects.begin(); '
+ 'ret && i != simObjects.end(); i ++)')
+ code('{')
+ code.indent()
+ code('${{param.ptype.cxx_type}} object = '
+ 'dynamic_cast<${{param.ptype.cxx_type}}>(*i);')
+ code('if (*i && !object)')
+ code(' ret = false;')
+ code('else')
+ code(' this->${{param.name}}.push_back(object);')
+ code.dedent()
+ code('}')
+ code.dedent()
+code.dedent()
+
+code('''
+ } else {
+ ret = false;
+ }
+
+ return ret;
+}
+
+void
+${param_class}::setName(const std::string &name_)
+{
+ this->name = name_;
+}
+
+bool
+${param_class}::setParam(const std::string &name,
+ const std::string &value, const Flags flags)
+{
+ bool ret = true;
+
+ if (false) {
+''')
+
+code.indent()
+for param in sim_object._params.values():
+ is_vector = isinstance(param, m5.params.VectorParamDesc)
+ is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
+
+ if not is_simobj and not is_vector:
+ code('} else if (name == "${{param.name}}") {')
+ code.indent()
+ param.ptype.cxx_ini_parse(code,
+ 'value', 'this->%s' % param.name, 'ret =')
+ code.dedent()
+code.dedent()
+
+code('''
+ } else {
+ ret = false;
+ }
+
+ return ret;
+}
+
+bool
+${param_class}::setParamVector(const std::string &name,
+ const std::vector<std::string> &values, const Flags flags)
+{
+ bool ret = true;
+
+ if (false) {
+''')
+
+code.indent()
+for param in sim_object._params.values():
+ is_vector = isinstance(param, m5.params.VectorParamDesc)
+ is_simobj = issubclass(param.ptype, m5.SimObject.SimObject)
+
+ if not is_simobj and is_vector:
+ code('} else if (name == "${{param.name}}") {')
+ code.indent()
+ code('${{param.name}}.clear();')
+ code('for (auto i = values.begin(); '
+ 'ret && i != values.end(); i ++)')
+ code('{')
+ code.indent()
+ code('${{param.ptype.cxx_type}} elem;')
+ param.ptype.cxx_ini_parse(code,
+ '*i', 'elem', 'ret =')
+ code('if (ret)')
+ code(' this->${{param.name}}.push_back(elem);')
+ code.dedent()
+ code('}')
+ code.dedent()
+code.dedent()
+
+code('''
+ } else {
+ ret = false;
+ }
+
+ return ret;
+}
+
+bool
+${param_class}::setPortConnectionCount(const std::string &name,
+ unsigned int count)
+{
+ bool ret = true;
+
+ if (false) {
+''')
+
+code.indent()
+for port in sim_object._ports.values():
+ code('} else if (name == "${{port.name}}") {')
+ code(' this->port_${{port.name}}_connection_count = count;')
+code.dedent()
+
+code('''
+ } else {
+ ret = false;
+ }
+
+ return ret;
+}
+
+SimObject *
+${param_class}::simObjectCreate()
+{
+''')
+
+code.indent()
+if hasattr(sim_object, 'abstract') and sim_object.abstract:
+ code('return nullptr;')
+else:
+ code('return this->create();')
+code.dedent()
+
+code('''}
+
+} // namespace gem5
+''')
+
+code.write(args.cxx_config_cc)
diff --git a/build_tools/cxx_config_hh.py b/build_tools/cxx_config_hh.py
new file mode 100644
index 0000000..652c488
--- /dev/null
+++ b/build_tools/cxx_config_hh.py
@@ -0,0 +1,115 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
+# Copyright 2021 Google, Inc.
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import argparse
+import importlib
+import os.path
+import sys
+
+import importer
+
+from code_formatter import code_formatter
+
+parser = argparse.ArgumentParser()
+parser.add_argument('modpath', help='module the simobject belongs to')
+parser.add_argument('cxx_config_hh', help='cxx config header file to generate')
+
+args = parser.parse_args()
+
+basename = os.path.basename(args.cxx_config_hh)
+sim_object_name = os.path.splitext(basename)[0]
+
+importer.install()
+module = importlib.import_module(args.modpath)
+sim_object = getattr(module, sim_object_name)
+
+code = code_formatter()
+
+entry_class = 'CxxConfigDirectoryEntry_%s' % sim_object_name
+param_class = '%sCxxConfigParams' % sim_object_name
+
+code('''#include "params/${sim_object_name}.hh"
+
+#include "sim/cxx_config.hh"
+
+namespace gem5
+{
+
+class ${param_class} : public CxxConfigParams, public ${sim_object_name}Params
+{
+ private:
+ class DirectoryEntry : public CxxConfigDirectoryEntry
+ {
+ public:
+ DirectoryEntry();
+
+ CxxConfigParams *
+ makeParamsObject() const
+ {
+ return new ${param_class};
+ }
+ };
+
+ static inline AddToConfigDir dirEntry
+ {"${sim_object_name}", new DirectoryEntry};
+
+ public:
+ bool setSimObject(const std::string &name, SimObject *simObject);
+
+ bool setSimObjectVector(const std::string &name,
+ const std::vector<SimObject *> &simObjects);
+
+ void setName(const std::string &name_);
+
+ const std::string &getName() { return this->name; }
+
+ bool setParam(const std::string &name, const std::string &value,
+ const Flags flags);
+
+ bool setParamVector(const std::string &name,
+ const std::vector<std::string> &values, const Flags flags);
+
+ bool setPortConnectionCount(const std::string &name, unsigned int count);
+
+ SimObject *simObjectCreate();
+};
+
+} // namespace gem5
+''')
+
+code.write(args.cxx_config_hh)
diff --git a/build_tools/enum_cc.py b/build_tools/enum_cc.py
new file mode 100644
index 0000000..c706ffe
--- /dev/null
+++ b/build_tools/enum_cc.py
@@ -0,0 +1,160 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
+# Copyright 2021 Google, Inc.
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import argparse
+import importlib
+import os.path
+import sys
+
+import importer
+
+from code_formatter import code_formatter
+
+parser = argparse.ArgumentParser()
+parser.add_argument('modpath', help='module the enum belongs to')
+parser.add_argument('enum_cc', help='enum cc file to generate')
+parser.add_argument('use_python',
+ help='whether python is enabled in gem5 (True or False)')
+
+args = parser.parse_args()
+
+use_python = args.use_python.lower()
+if use_python == 'true':
+ use_python = True
+elif use_python == 'false':
+ use_python = False
+else:
+ print(f'Unrecognized "use_python" value {use_python}', file=sys.stderr)
+ sys.exit(1)
+
+basename = os.path.basename(args.enum_cc)
+enum_name = os.path.splitext(basename)[0]
+
+importer.install()
+module = importlib.import_module(args.modpath)
+enum = getattr(module, enum_name)
+
+code = code_formatter()
+
+wrapper_name = enum.wrapper_name
+file_name = enum.__name__
+name = enum.__name__ if enum.enum_name is None else enum.enum_name
+
+code('''#include "base/compiler.hh"
+#include "enums/$file_name.hh"
+
+namespace gem5
+{
+
+''')
+
+if enum.wrapper_is_struct:
+ code('const char *${wrapper_name}::${name}Strings'
+ '[Num_${name}] =')
+else:
+ if enum.is_class:
+ code('''\
+const char *${name}Strings[static_cast<int>(${name}::Num_${name})] =
+''')
+ else:
+ code('''GEM5_DEPRECATED_NAMESPACE(Enums, enums);
+namespace enums
+{''')
+ code.indent(1)
+ code('const char *${name}Strings[Num_${name}] =')
+
+code('{')
+code.indent(1)
+for val in enum.vals:
+ code('"$val",')
+code.dedent(1)
+code('};')
+
+if not enum.wrapper_is_struct and not enum.is_class:
+ code.dedent(1)
+ code('} // namespace enums')
+
+code('} // namespace gem5')
+
+
+if use_python:
+
+ name = enum.__name__
+ enum_name = enum.__name__ if enum.enum_name is None else enum.enum_name
+ wrapper_name = enum_name if enum.is_class else enum.wrapper_name
+
+ code('''#include "pybind11/pybind11.h"
+#include "pybind11/stl.h"
+
+#include <sim/init.hh>
+
+namespace py = pybind11;
+
+namespace gem5
+{
+
+static void
+module_init(py::module_ &m_internal)
+{
+ py::module_ m = m_internal.def_submodule("enum_${name}");
+
+''')
+ if enum.is_class:
+ code('py::enum_<${enum_name}>(m, "enum_${name}")')
+ else:
+ code('py::enum_<${wrapper_name}::${enum_name}>(m, "enum_${name}")')
+
+ code.indent()
+ code.indent()
+ for val in enum.vals:
+ code('.value("${val}", ${wrapper_name}::${val})')
+ code('.value("Num_${name}", ${wrapper_name}::Num_${enum_name})')
+ if not enum.is_class:
+ code('.export_values()')
+ code(';')
+ code.dedent()
+
+ code('}')
+ code.dedent()
+ code('''
+static EmbeddedPyBind embed_enum("enum_${name}", module_init);
+
+} // namespace gem5
+ ''')
+
+code.write(args.enum_cc)
diff --git a/build_tools/enum_hh.py b/build_tools/enum_hh.py
new file mode 100644
index 0000000..2c4a7bb
--- /dev/null
+++ b/build_tools/enum_hh.py
@@ -0,0 +1,116 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
+# Copyright 2021 Google, Inc.
+#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import argparse
+import importlib
+import os.path
+import sys
+
+import importer
+
+from code_formatter import code_formatter
+
+parser = argparse.ArgumentParser()
+parser.add_argument('modpath', help='module the enum belongs to')
+parser.add_argument('enum_hh', help='enum header file to generate')
+
+args = parser.parse_args()
+
+basename = os.path.basename(args.enum_hh)
+enum_name = os.path.splitext(basename)[0]
+
+importer.install()
+module = importlib.import_module(args.modpath)
+enum = getattr(module, enum_name)
+
+code = code_formatter()
+
+# Generate C++ class declaration for this enum type.
+# Note that we wrap the enum in a class/struct to act as a namespace,
+# so that the enum strings can be brief w/o worrying about collisions.
+wrapper_name = enum.wrapper_name
+wrapper = 'struct' if enum.wrapper_is_struct else 'namespace'
+name = enum.__name__ if enum.enum_name is None else enum.enum_name
+idem_macro = '__ENUM__%s__%s__' % (wrapper_name, name)
+
+code('''\
+#ifndef $idem_macro
+#define $idem_macro
+
+namespace gem5
+{
+''')
+if enum.is_class:
+ code('''\
+enum class $name
+{
+''')
+else:
+ code('''\
+$wrapper $wrapper_name {
+enum $name
+{
+''')
+ code.indent(1)
+code.indent(1)
+for val in enum.vals:
+ code('$val = ${{enum.map[val]}},')
+code('Num_$name = ${{len(enum.vals)}}')
+code.dedent(1)
+code('};')
+
+if enum.is_class:
+ code('''\
+extern const char *${name}Strings[static_cast<int>(${name}::Num_${name})];
+''')
+elif enum.wrapper_is_struct:
+ code('static const char *${name}Strings[Num_${name}];')
+else:
+ code('extern const char *${name}Strings[Num_${name}];')
+
+if not enum.is_class:
+ code.dedent(1)
+ code('}; // $wrapper_name')
+
+code()
+code('} // namespace gem5')
+
+code()
+code('#endif // $idem_macro')
+
+code.write(args.enum_hh)
diff --git a/build_tools/sim_object_param_struct_cc.py b/build_tools/sim_object_param_struct_cc.py
index 7266556..1b72e3c 100644
--- a/build_tools/sim_object_param_struct_cc.py
+++ b/build_tools/sim_object_param_struct_cc.py
@@ -1,5 +1,18 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
@@ -57,6 +70,204 @@
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
+from m5.objects.SimObject import PyBindProperty
+
code = code_formatter()
-sim_object.params_create_decl(code, use_python)
+
+py_class_name = sim_object.pybind_class
+
+# The 'local' attribute restricts us to the params declared in
+# the object itself, not including inherited params (which
+# will also be inherited from the base class's param struct
+# here). Sort the params based on their key
+params = list(map(lambda k_v: k_v[1],
+ sorted(sim_object._params.local.items())))
+ports = sim_object._ports.local
+
+# only include pybind if python is enabled in the build
+if use_python:
+
+ code('''#include "pybind11/pybind11.h"
+#include "pybind11/stl.h"
+
+#include <type_traits>
+
+#include "base/compiler.hh"
+#include "params/$sim_object.hh"
+#include "sim/init.hh"
+#include "sim/sim_object.hh"
+
+#include "${{sim_object.cxx_header}}"
+
+''')
+else:
+ code('''
+#include <type_traits>
+
+#include "base/compiler.hh"
+#include "params/$sim_object.hh"
+
+#include "${{sim_object.cxx_header}}"
+
+''')
+# only include the python params code if python is enabled.
+if use_python:
+ for param in params:
+ param.pybind_predecls(code)
+
+ code('''namespace py = pybind11;
+
+namespace gem5
+{
+
+static void
+module_init(py::module_ &m_internal)
+{
+py::module_ m = m_internal.def_submodule("param_${sim_object}");
+''')
+ code.indent()
+ if sim_object._base:
+ code('py::class_<${sim_object}Params, ' \
+ '${{sim_object._base.type}}Params, ' \
+ 'std::unique_ptr<${{sim_object}}Params, py::nodelete>>(' \
+ 'm, "${sim_object}Params")')
+ else:
+ code('py::class_<${sim_object}Params, ' \
+ 'std::unique_ptr<${sim_object}Params, py::nodelete>>(' \
+ 'm, "${sim_object}Params")')
+
+ code.indent()
+ if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
+ code('.def(py::init<>())')
+ code('.def("create", &${sim_object}Params::create)')
+
+ param_exports = sim_object.cxx_param_exports + [
+ PyBindProperty(k)
+ for k, v in sorted(sim_object._params.local.items())
+ ] + [
+ PyBindProperty(f"port_{port.name}_connection_count")
+ for port in ports.values()
+ ]
+ for exp in param_exports:
+ exp.export(code, f"{sim_object}Params")
+
+ code(';')
+ code()
+ code.dedent()
+
+ bases = []
+ if 'cxx_base' in sim_object._value_dict:
+ # If the c++ base class implied by python inheritance was
+ # overridden, use that value.
+ if sim_object.cxx_base:
+ bases.append(sim_object.cxx_base)
+ elif sim_object._base:
+ # If not and if there was a SimObject base, use its c++ class
+ # as this class' base.
+ bases.append(sim_object._base.cxx_class)
+ # Add in any extra bases that were requested.
+ bases.extend(sim_object.cxx_extra_bases)
+
+ if bases:
+ base_str = ", ".join(bases)
+ code('py::class_<${{sim_object.cxx_class}}, ${base_str}, ' \
+ 'std::unique_ptr<${{sim_object.cxx_class}}, py::nodelete>>(' \
+ 'm, "${py_class_name}")')
+ else:
+ code('py::class_<${{sim_object.cxx_class}}, ' \
+ 'std::unique_ptr<${{sim_object.cxx_class}}, py::nodelete>>(' \
+ 'm, "${py_class_name}")')
+ code.indent()
+ for exp in sim_object.cxx_exports:
+ exp.export(code, sim_object.cxx_class)
+ code(';')
+ code.dedent()
+ code()
+ code.dedent()
+ code('}')
+ code()
+ code('static EmbeddedPyBind '
+ 'embed_obj("${0}", module_init, "${1}");',
+ sim_object, sim_object._base.type if sim_object._base else "")
+ code()
+ code('} // namespace gem5')
+
+# include the create() methods whether or not python is enabled.
+if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
+ if 'type' in sim_object.__dict__:
+ code('''
+namespace gem5
+{
+
+namespace
+{
+
+/*
+ * If we can't define a default create() method for this params
+ * struct because the SimObject doesn't have the right
+ * constructor, use template magic to make it so we're actually
+ * defining a create method for this class instead.
+ */
+class Dummy${sim_object}ParamsClass
+{
+ public:
+ ${{sim_object.cxx_class}} *create() const;
+};
+
+template <class CxxClass, class Enable=void>
+class Dummy${sim_object}Shunt;
+
+/*
+ * This version directs to the real Params struct and the
+ * default behavior of create if there's an appropriate
+ * constructor.
+ */
+template <class CxxClass>
+class Dummy${sim_object}Shunt<CxxClass, std::enable_if_t<
+ std::is_constructible_v<CxxClass, const ${sim_object}Params &>>>
+{
+ public:
+ using Params = ${sim_object}Params;
+ static ${{sim_object.cxx_class}} *
+ create(const Params &p)
+ {
+ return new CxxClass(p);
+ }
+};
+
+/*
+ * This version diverts to the DummyParamsClass and a dummy
+ * implementation of create if the appropriate constructor does
+ * not exist.
+ */
+template <class CxxClass>
+class Dummy${sim_object}Shunt<CxxClass, std::enable_if_t<
+ !std::is_constructible_v<CxxClass, const ${sim_object}Params &>>>
+{
+ public:
+ using Params = Dummy${sim_object}ParamsClass;
+ static ${{sim_object.cxx_class}} *
+ create(const Params &p)
+ {
+ return nullptr;
+ }
+};
+
+} // anonymous namespace
+
+/*
+ * An implementation of either the real Params struct's create
+ * method, or the Dummy one. Either an implementation is
+ * mandantory since this was shunted off to the dummy class, or
+ * one is optional which will override this weak version.
+ */
+[[maybe_unused]] ${{sim_object.cxx_class}} *
+Dummy${sim_object}Shunt<${{sim_object.cxx_class}}>::Params::create() const
+{
+ return Dummy${sim_object}Shunt<${{sim_object.cxx_class}}>::create(*this);
+}
+
+} // namespace gem5
+''')
+
code.write(args.param_cc)
diff --git a/build_tools/sim_object_param_struct_hh.py b/build_tools/sim_object_param_struct_hh.py
index 0652ae5..261ac9b 100644
--- a/build_tools/sim_object_param_struct_hh.py
+++ b/build_tools/sim_object_param_struct_hh.py
@@ -1,5 +1,18 @@
+# Copyright 2004-2006 The Regents of The University of Michigan
+# Copyright 2010-20013 Advanced Micro Devices, Inc.
+# Copyright 2013 Mark D. Hill and David A. Wood
+# Copyright 2017-2020 ARM Limited
# Copyright 2021 Google, Inc.
#
+# The license below extends only to copyright in the software and shall
+# not be construed as granting a license to any other intellectual
+# property including but not limited to intellectual property relating
+# to a hardware implementation of the functionality of the software
+# licensed hereunder. You may use the software subject to the license
+# terms below provided that you ensure that this notice is replicated
+# unmodified and in its entirety in all distributions of the software,
+# modified or unmodified, in source code or in binary form.
+#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
@@ -45,6 +58,167 @@
module = importlib.import_module(args.modpath)
sim_object = getattr(module, sim_object_name)
+from m5.objects.SimObject import SimObject
+from m5.params import Enum
+
code = code_formatter()
-sim_object.cxx_param_decl(code)
+
+# The 'local' attribute restricts us to the params declared in
+# the object itself, not including inherited params (which
+# will also be inherited from the base class's param struct
+# here). Sort the params based on their key
+params = list(map(lambda k_v: k_v[1],
+ sorted(sim_object._params.local.items())))
+ports = sim_object._ports.local
+try:
+ ptypes = [p.ptype for p in params]
+except:
+ print(sim_object)
+ print(params)
+ raise
+
+warned_about_nested_templates = False
+
+class CxxClass(object):
+ def __init__(self, sig, template_params=[]):
+ # Split the signature into its constituent parts. This could
+ # potentially be done with regular expressions, but
+ # it's simple enough to pick appart a class signature
+ # manually.
+ parts = sig.split('<', 1)
+ base = parts[0]
+ t_args = []
+ if len(parts) > 1:
+ # The signature had template arguments.
+ text = parts[1].rstrip(' \t\n>')
+ arg = ''
+ # Keep track of nesting to avoid splitting on ","s embedded
+ # in the arguments themselves.
+ depth = 0
+ for c in text:
+ if c == '<':
+ depth = depth + 1
+ if depth > 0 and not warned_about_nested_templates:
+ warned_about_nested_templates = True
+ print('Nested template argument in cxx_class.'
+ ' This feature is largely untested and '
+ ' may not work.')
+ elif c == '>':
+ depth = depth - 1
+ elif c == ',' and depth == 0:
+ t_args.append(arg.strip())
+ arg = ''
+ else:
+ arg = arg + c
+ if arg:
+ t_args.append(arg.strip())
+ # Split the non-template part on :: boundaries.
+ class_path = base.split('::')
+
+ # The namespaces are everything except the last part of the class path.
+ self.namespaces = class_path[:-1]
+ # And the class name is the last part.
+ self.name = class_path[-1]
+
+ self.template_params = template_params
+ self.template_arguments = []
+ # Iterate through the template arguments and their values. This
+ # will likely break if parameter packs are used.
+ for arg, param in zip(t_args, template_params):
+ type_keys = ('class', 'typename')
+ # If a parameter is a type, parse it recursively. Otherwise
+ # assume it's a constant, and store it verbatim.
+ if any(param.strip().startswith(kw) for kw in type_keys):
+ self.template_arguments.append(CxxClass(arg))
+ else:
+ self.template_arguments.append(arg)
+
+ def declare(self, code):
+ # First declare any template argument types.
+ for arg in self.template_arguments:
+ if isinstance(arg, CxxClass):
+ arg.declare(code)
+ # Re-open the target namespace.
+ for ns in self.namespaces:
+ code('namespace $ns {')
+ # If this is a class template...
+ if self.template_params:
+ code('template <${{", ".join(self.template_params)}}>')
+ # The actual class declaration.
+ code('class ${{self.name}};')
+ # Close the target namespaces.
+ for ns in reversed(self.namespaces):
+ code('} // namespace $ns')
+
+code('''\
+#ifndef __PARAMS__${sim_object}__
+#define __PARAMS__${sim_object}__
+
+''')
+
+
+# The base SimObject has a couple of params that get
+# automatically set from Python without being declared through
+# the normal Param mechanism; we slip them in here (needed
+# predecls now, actual declarations below)
+if sim_object == SimObject:
+ code('''#include <string>''')
+
+cxx_class = CxxClass(sim_object._value_dict['cxx_class'],
+ sim_object._value_dict['cxx_template_params'])
+
+# A forward class declaration is sufficient since we are just
+# declaring a pointer.
+cxx_class.declare(code)
+
+for param in params:
+ param.cxx_predecls(code)
+for port in ports.values():
+ port.cxx_predecls(code)
+code()
+
+if sim_object._base:
+ code('#include "params/${{sim_object._base.type}}.hh"')
+ code()
+
+for ptype in ptypes:
+ if issubclass(ptype, Enum):
+ code('#include "enums/${{ptype.__name__}}.hh"')
+ code()
+
+code('namespace gem5')
+code('{')
+code('')
+
+# now generate the actual param struct
+code("struct ${sim_object}Params")
+if sim_object._base:
+ code(" : public ${{sim_object._base.type}}Params")
+code("{")
+if not hasattr(sim_object, 'abstract') or not sim_object.abstract:
+ if 'type' in sim_object.__dict__:
+ code(" ${{sim_object.cxx_type}} create() const;")
+
+code.indent()
+if sim_object == SimObject:
+ code('''
+SimObjectParams() {}
+virtual ~SimObjectParams() {}
+
+std::string name;
+ ''')
+
+for param in params:
+ param.cxx_decl(code)
+for port in ports.values():
+ port.cxx_decl(code)
+
+code.dedent()
+code('};')
+code()
+code('} // namespace gem5')
+
+code()
+code('#endif // __PARAMS__${sim_object}__')
+
code.write(args.param_hh)
diff --git a/configs/common/FSConfig.py b/configs/common/FSConfig.py
index 8b8fb4e..febe146 100644
--- a/configs/common/FSConfig.py
+++ b/configs/common/FSConfig.py
@@ -74,8 +74,7 @@
viodir = os.path.realpath(os.path.join(m5.options.outdir, '9p'))
viopci.vio.root = os.path.join(viodir, 'share')
viopci.vio.socketPath = os.path.join(viodir, 'socket')
- if not os.path.exists(viopci.vio.root):
- os.makedirs(viopci.vio.root)
+ os.makedirs(viopci.vio.root, exist_ok=True)
if os.path.exists(viopci.vio.socketPath):
os.remove(viopci.vio.socketPath)
parent.viopci = viopci
@@ -445,6 +444,8 @@
def makeX86System(mem_mode, numCPUs=1, mdesc=None, workload=None, Ruby=False):
self = System()
+ self.m5ops_base = 0xffff0000
+
if workload is None:
workload = X86FsWorkload()
self.workload = workload
diff --git a/configs/common/FileSystemConfig.py b/configs/common/FileSystemConfig.py
index 66a6315..f60bf23 100644
--- a/configs/common/FileSystemConfig.py
+++ b/configs/common/FileSystemConfig.py
@@ -140,7 +140,7 @@
# Set up /sys/devices/system/cpu
cpudir = joinpath(sysdir, 'devices', 'system', 'cpu')
- makedirs(cpudir)
+ makedirs(cpudir, exist_ok=True)
file_append((cpudir, 'online'), '0-%d' % (len(cpus) - 1))
file_append((cpudir, 'possible'), '0-%d' % (len(cpus) - 1))
@@ -168,7 +168,7 @@
'system', 'node')
nodedir = joinpath(nodebasedir,'node%d' % node_number)
- makedirs(nodedir)
+ makedirs(nodedir, exist_ok=True)
file_append((nodedir, 'cpumap'), hex_mask(cpu_list))
file_append((nodedir, 'meminfo'),
@@ -180,10 +180,8 @@
cpudir = joinpath(m5.options.outdir, 'fs', 'sys', 'devices', 'system',
'cpu', 'cpu%d' % core_id)
- if not isdir(joinpath(cpudir, 'topology')):
- makedirs(joinpath(cpudir, 'topology'))
- if not isdir(joinpath(cpudir, 'cache')):
- makedirs(joinpath(cpudir, 'cache'))
+ makedirs(joinpath(cpudir, 'topology'), exist_ok=True)
+ makedirs(joinpath(cpudir, 'cache'))
file_append((cpudir, 'online'), '1')
file_append((cpudir, 'topology', 'physical_package_id'),
@@ -204,7 +202,7 @@
while isdir(joinpath(cachedir, 'index%d' % j)):
j += 1
indexdir = joinpath(cachedir, 'index%d' % j)
- makedirs(indexdir)
+ makedirs(indexdir, exist_ok=True)
file_append((indexdir, 'level'), level)
file_append((indexdir, 'type'), idu_type)
diff --git a/configs/common/GPUTLBConfig.py b/configs/common/GPUTLBConfig.py
index f6aaccf..740c748 100644
--- a/configs/common/GPUTLBConfig.py
+++ b/configs/common/GPUTLBConfig.py
@@ -34,41 +34,69 @@
import m5
from m5.objects import *
-def TLB_constructor(level):
+def TLB_constructor(options, level, gpu_ctrl=None, full_system=False):
- constructor_call = "X86GPUTLB(size = options.L%(level)dTLBentries, \
- assoc = options.L%(level)dTLBassoc, \
- hitLatency = options.L%(level)dAccessLatency,\
- missLatency2 = options.L%(level)dMissLatency,\
- maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
- accessDistance = options.L%(level)dAccessDistanceStat,\
- clk_domain = SrcClockDomain(\
- clock = options.gpu_clock,\
- voltage_domain = VoltageDomain(\
- voltage = options.gpu_voltage)))" % locals()
- return constructor_call
-
-def Coalescer_constructor(level):
-
- constructor_call = "TLBCoalescer(probesPerCycle = \
- options.L%(level)dProbesPerCycle, \
- coalescingWindow = options.L%(level)dCoalescingWindow,\
- disableCoalescing = options.L%(level)dDisableCoalescing,\
+ if full_system:
+ constructor_call = "VegaGPUTLB(\
+ gpu_device = gpu_ctrl, \
+ size = options.L%(level)dTLBentries, \
+ assoc = options.L%(level)dTLBassoc, \
+ hitLatency = options.L%(level)dAccessLatency,\
+ missLatency1 = options.L%(level)dMissLatency,\
+ missLatency2 = options.L%(level)dMissLatency,\
+ maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
+ clk_domain = SrcClockDomain(\
+ clock = options.gpu_clock,\
+ voltage_domain = VoltageDomain(\
+ voltage = options.gpu_voltage)))" % locals()
+ else:
+ constructor_call = "X86GPUTLB(size = options.L%(level)dTLBentries, \
+ assoc = options.L%(level)dTLBassoc, \
+ hitLatency = options.L%(level)dAccessLatency,\
+ missLatency2 = options.L%(level)dMissLatency,\
+ maxOutstandingReqs = options.L%(level)dMaxOutstandingReqs,\
+ accessDistance = options.L%(level)dAccessDistanceStat,\
clk_domain = SrcClockDomain(\
clock = options.gpu_clock,\
voltage_domain = VoltageDomain(\
voltage = options.gpu_voltage)))" % locals()
return constructor_call
+def Coalescer_constructor(options, level, full_system):
+
+ if full_system:
+ constructor_call = "VegaTLBCoalescer(probesPerCycle = \
+ options.L%(level)dProbesPerCycle, \
+ tlb_level = %(level)d ,\
+ coalescingWindow = options.L%(level)dCoalescingWindow,\
+ disableCoalescing = options.L%(level)dDisableCoalescing,\
+ clk_domain = SrcClockDomain(\
+ clock = options.gpu_clock,\
+ voltage_domain = VoltageDomain(\
+ voltage = options.gpu_voltage)))" % locals()
+ else:
+ constructor_call = "TLBCoalescer(probesPerCycle = \
+ options.L%(level)dProbesPerCycle, \
+ coalescingWindow = options.L%(level)dCoalescingWindow,\
+ disableCoalescing = options.L%(level)dDisableCoalescing,\
+ clk_domain = SrcClockDomain(\
+ clock = options.gpu_clock,\
+ voltage_domain = VoltageDomain(\
+ voltage = options.gpu_voltage)))" % locals()
+ return constructor_call
+
def create_TLB_Coalescer(options, my_level, my_index, tlb_name,
- coalescer_name):
+ coalescer_name, gpu_ctrl=None, full_system=False):
# arguments: options, TLB level, number of private structures for this
# Level, TLB name and Coalescer name
for i in range(my_index):
- tlb_name.append(eval(TLB_constructor(my_level)))
- coalescer_name.append(eval(Coalescer_constructor(my_level)))
+ tlb_name.append(
+ eval(TLB_constructor(options, my_level, gpu_ctrl, full_system)))
+ coalescer_name.append(
+ eval(Coalescer_constructor(options, my_level, full_system)))
-def config_tlb_hierarchy(options, system, shader_idx):
+def config_tlb_hierarchy(options, system, shader_idx, gpu_ctrl=None,
+ full_system=False):
n_cu = options.num_compute_units
if options.TLB_config == "perLane":
@@ -121,7 +149,7 @@
options.L1TLBassoc = options.L1TLBentries
# call the constructors for the TLB and the Coalescer
create_TLB_Coalescer(options, level, TLB_index,\
- TLB_array, Coalescer_array)
+ TLB_array, Coalescer_array, gpu_ctrl, full_system)
system_TLB_name = TLB_type['name'] + '_tlb'
system_Coalescer_name = TLB_type['name'] + '_coalescer'
@@ -198,8 +226,17 @@
system.l2_coalescer[0].cpu_side_ports[%d]' % \
(name, index, l2_coalescer_index))
l2_coalescer_index += 1
+
# L2 <-> L3
system.l2_tlb[0].mem_side_ports[0] = \
system.l3_coalescer[0].cpu_side_ports[0]
+ # L3 TLB Vega page table walker to memory for full system only
+ if full_system:
+ for TLB_type in L3:
+ name = TLB_type['name']
+ for index in range(TLB_type['width']):
+ exec('system._dma_ports.append(system.%s_tlb[%d].walker)' % \
+ (name, index))
+
return system
diff --git a/configs/common/MemConfig.py b/configs/common/MemConfig.py
index 15af26f..332fd6b 100644
--- a/configs/common/MemConfig.py
+++ b/configs/common/MemConfig.py
@@ -235,7 +235,7 @@
# Create a controller if not sharing a channel with DRAM
# in which case the controller has already been created
if not opt_hybrid_channel:
- mem_ctrl = m5.objects.MemCtrl()
+ mem_ctrl = m5.objects.HeteroMemCtrl()
mem_ctrl.nvm = nvm_intf
mem_ctrls.append(mem_ctrl)
diff --git a/configs/common/Simulation.py b/configs/common/Simulation.py
index 3b9efc0..2416773 100644
--- a/configs/common/Simulation.py
+++ b/configs/common/Simulation.py
@@ -488,6 +488,7 @@
options.indirect_bp_type)
switch_cpus[i].branchPred.indirectBranchPred = \
IndirectBPClass()
+ switch_cpus[i].createThreads()
# If elastic tracing is enabled attach the elastic trace probe
# to the switch CPUs
diff --git a/configs/common/cores/arm/HPI.py b/configs/common/cores/arm/HPI.py
index 620c01e..3a11133 100644
--- a/configs/common/cores/arm/HPI.py
+++ b/configs/common/cores/arm/HPI.py
@@ -1379,7 +1379,7 @@
write_buffers = 16
# prefetcher FIXME
-class HPI(MinorCPU):
+class HPI(ArmMinorCPU):
# Inherit the doc string from the module to avoid repeating it
# here.
__doc__ = __doc__
diff --git a/configs/common/cores/arm/O3_ARM_v7a.py b/configs/common/cores/arm/O3_ARM_v7a.py
index 8cacc65..d032a1a 100644
--- a/configs/common/cores/arm/O3_ARM_v7a.py
+++ b/configs/common/cores/arm/O3_ARM_v7a.py
@@ -99,7 +99,7 @@
RASSize = 16
instShiftAmt = 2
-class O3_ARM_v7a_3(DerivO3CPU):
+class O3_ARM_v7a_3(ArmO3CPU):
LQEntries = 16
SQEntries = 16
LSQDepCheckShift = 0
diff --git a/configs/common/cores/arm/ex5_LITTLE.py b/configs/common/cores/arm/ex5_LITTLE.py
index bcbaa92..57f6a6b 100644
--- a/configs/common/cores/arm/ex5_LITTLE.py
+++ b/configs/common/cores/arm/ex5_LITTLE.py
@@ -88,7 +88,7 @@
ex5_LITTLE_FP(), ex5_LITTLE_MemFU(),
ex5_LITTLE_MiscFU()]
-class ex5_LITTLE(MinorCPU):
+class ex5_LITTLE(ArmMinorCPU):
executeFuncUnits = ex5_LITTLE_FUP()
class L1Cache(Cache):
diff --git a/configs/common/cores/arm/ex5_big.py b/configs/common/cores/arm/ex5_big.py
index eb5f53f..de7a450 100644
--- a/configs/common/cores/arm/ex5_big.py
+++ b/configs/common/cores/arm/ex5_big.py
@@ -99,7 +99,7 @@
RASSize = 48
instShiftAmt = 2
-class ex5_big(DerivO3CPU):
+class ex5_big(ArmO3CPU):
LQEntries = 16
SQEntries = 16
LSQDepCheckShift = 0
diff --git a/configs/example/apu_se.py b/configs/example/apu_se.py
index 882d181..b5fb9ff 100644
--- a/configs/example/apu_se.py
+++ b/configs/example/apu_se.py
@@ -161,7 +161,7 @@
' m5_switchcpu pseudo-ops will toggle back and forth')
parser.add_argument("--num-hw-queues", type=int, default=10,
help="number of hw queues in packet processor")
-parser.add_argument("--reg-alloc-policy", type=str, default="simple",
+parser.add_argument("--reg-alloc-policy", type=str, default="dynamic",
help="register allocation policy (simple/dynamic)")
parser.add_argument("--dgpu", action="store_true", default=False,
@@ -544,6 +544,7 @@
have_kvm_support = 'BaseKvmCPU' in globals()
if have_kvm_support and buildEnv['TARGET_ISA'] == "x86":
system.vm = KvmVM()
+ system.m5ops_base = 0xffff0000
for i in range(len(host_cpu.workload)):
host_cpu.workload[i].useArchPT = True
host_cpu.workload[i].kvmInSE = True
diff --git a/configs/example/arm/baremetal.py b/configs/example/arm/baremetal.py
index 0944344..44e3fd1 100644
--- a/configs/example/arm/baremetal.py
+++ b/configs/example/arm/baremetal.py
@@ -141,8 +141,6 @@
system.realview.gic.gicv4 = False
system.highest_el_is_64 = True
- system.release.add(ArmExtension('SECURITY'))
- system.release.add(ArmExtension('VIRTUALIZATION'))
workload_class = workloads.workload_list.get(args.workload)
system.workload = workload_class(
diff --git a/configs/example/arm/devices.py b/configs/example/arm/devices.py
index 9122e7c..a488ab3 100644
--- a/configs/example/arm/devices.py
+++ b/configs/example/arm/devices.py
@@ -176,7 +176,7 @@
class AtomicCluster(CpuCluster):
def __init__(self, system, num_cpus, cpu_clock, cpu_voltage="1.0V"):
cpu_config = [ ObjectList.cpu_list.get("AtomicSimpleCPU"), None,
- None, None, None ]
+ None, None ]
super(AtomicCluster, self).__init__(system, num_cpus, cpu_clock,
cpu_voltage, *cpu_config)
def addL1(self):
@@ -185,7 +185,7 @@
class KvmCluster(CpuCluster):
def __init__(self, system, num_cpus, cpu_clock, cpu_voltage="1.0V"):
cpu_config = [ ObjectList.cpu_list.get("ArmV8KvmCPU"), None, None,
- None, None ]
+ None ]
super(KvmCluster, self).__init__(system, num_cpus, cpu_clock,
cpu_voltage, *cpu_config)
def addL1(self):
diff --git a/configs/example/arm/ruby_fs.py b/configs/example/arm/ruby_fs.py
index d820f86..ecca22b 100644
--- a/configs/example/arm/ruby_fs.py
+++ b/configs/example/arm/ruby_fs.py
@@ -1,4 +1,4 @@
-# Copyright (c) 2016-2017, 2020-2021 Arm Limited
+# Copyright (c) 2016-2017, 2020-2022 Arm Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
@@ -46,7 +46,7 @@
from common import ObjectList
from common import Options
from common import SysPaths
-from common.cores.arm import HPI
+from common.cores.arm import O3_ARM_v7a, HPI
from ruby import Ruby
import devices
@@ -57,18 +57,12 @@
default_root_device = '/dev/vda1'
-# Pre-defined CPU configurations. Each tuple must be ordered as : (cpu_class,
-# l1_icache_class, l1_dcache_class, walk_cache_class, l2_Cache_class). Any of
-# the cache class may be 'None' if the particular cache is not present.
+# Pre-defined CPU configurations.
cpu_types = {
-
- "noncaching" : ( NonCachingSimpleCPU, None, None, None),
- "minor" : (MinorCPU,
- devices.L1I, devices.L1D,
- devices.L2),
- "hpi" : ( HPI.HPI,
- HPI.HPI_ICache, HPI.HPI_DCache,
- HPI.HPI_L2)
+ "noncaching" : NonCachingSimpleCPU,
+ "minor" : MinorCPU,
+ "hpi" : HPI.HPI,
+ "o3" : O3_ARM_v7a.O3_ARM_v7a_3,
}
def create_cow_image(name):
@@ -100,7 +94,7 @@
print("Error: Bootscript %s does not exist" % args.script)
sys.exit(1)
- cpu_class = cpu_types[args.cpu][0]
+ cpu_class = cpu_types[args.cpu]
mem_mode = cpu_class.memory_mode()
system = devices.ArmRubySystem(args.mem_size,
@@ -115,7 +109,7 @@
devices.CpuCluster(system,
args.num_cpus,
args.cpu_freq, "1.0V",
- *cpu_types[args.cpu]),
+ cpu_class, None, None, None),
]
# Add the PCI devices we need for this system. The base system
diff --git a/configs/example/arm/starter_fs.py b/configs/example/arm/starter_fs.py
index 40e645b..140f102 100644
--- a/configs/example/arm/starter_fs.py
+++ b/configs/example/arm/starter_fs.py
@@ -1,4 +1,4 @@
-# Copyright (c) 2016-2017, 2020 ARM Limited
+# Copyright (c) 2016-2017, 2020, 2022 Arm Limited
# All rights reserved.
#
# The license below extends only to copyright in the software and shall
@@ -50,7 +50,7 @@
from common import SysPaths
from common import ObjectList
from common import MemConfig
-from common.cores.arm import HPI
+from common.cores.arm import O3_ARM_v7a, HPI
import devices
@@ -61,17 +61,19 @@
# Pre-defined CPU configurations. Each tuple must be ordered as : (cpu_class,
-# l1_icache_class, l1_dcache_class, walk_cache_class, l2_Cache_class). Any of
+# l1_icache_class, l1_dcache_class, l2_Cache_class). Any of
# the cache class may be 'None' if the particular cache is not present.
cpu_types = {
-
- "atomic" : ( AtomicSimpleCPU, None, None, None),
+ "atomic" : (AtomicSimpleCPU, None, None, None),
"minor" : (MinorCPU,
devices.L1I, devices.L1D,
devices.L2),
- "hpi" : ( HPI.HPI,
- HPI.HPI_ICache, HPI.HPI_DCache,
- HPI.HPI_L2)
+ "hpi" : (HPI.HPI,
+ HPI.HPI_ICache, HPI.HPI_DCache,
+ HPI.HPI_L2),
+ "o3" : (O3_ARM_v7a.O3_ARM_v7a_3,
+ O3_ARM_v7a.O3_ARM_v7a_ICache, O3_ARM_v7a.O3_ARM_v7a_DCache,
+ O3_ARM_v7a.O3_ARM_v7aL2),
}
def create_cow_image(name):
@@ -148,6 +150,9 @@
os.path.join(m5.options.outdir, 'system.dtb')
system.generateDtb(system.workload.dtb_filename)
+ if args.initrd:
+ system.workload.initrd_filename = args.initrd
+
# Linux boot command flags
kernel_cmd = [
# Tell Linux to use the simulated serial port as a console
@@ -196,6 +201,8 @@
help="DTB file to load")
parser.add_argument("--kernel", type=str, default=default_kernel,
help="Linux kernel")
+ parser.add_argument("--initrd", type=str, default=None,
+ help="initrd/initramfs file to load")
parser.add_argument("--disk-image", type=str,
default=default_disk,
help="Disk to instantiate")
diff --git a/configs/example/gem5_library/arm-hello.py b/configs/example/gem5_library/arm-hello.py
index b1f6f38..d94fb33 100644
--- a/configs/example/gem5_library/arm-hello.py
+++ b/configs/example/gem5_library/arm-hello.py
@@ -37,7 +37,7 @@
```
scons build/ARM/gem5.opt
-./build/ARM/gem5.opt configs/gem5_library/arm-hello.py
+./build/ARM/gem5.opt configs/example/gem5_library/arm-hello.py
```
"""
@@ -62,7 +62,7 @@
memory = SingleChannelDDR3_1600(size="32MB")
# We use a simple Timing processor with one core.
-processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, num_cores=1)
+processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.ARM, num_cores=1)
# The gem5 library simble board which can be used to run simple SE-mode
# simulations.
@@ -88,7 +88,7 @@
)
# Lastly we run the simulation.
-simulator = Simulator(board=board, full_system=False)
+simulator = Simulator(board=board)
simulator.run()
print(
diff --git a/configs/example/gem5_library/arm-ubuntu-boot-exit.py b/configs/example/gem5_library/arm-ubuntu-boot-exit.py
new file mode 100644
index 0000000..163f45a
--- /dev/null
+++ b/configs/example/gem5_library/arm-ubuntu-boot-exit.py
@@ -0,0 +1,152 @@
+# Copyright (c) 2022 The Regents of the University of California
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+This script shows an example of booting an ARM based full system Ubuntu
+disk image using the gem5's standard library. This simulation boots the disk
+image using 2 TIMING CPU cores. The simulation ends when the startup is
+completed successfully (i.e. when an `m5_exit instruction is reached on
+successful boot).
+
+Usage
+-----
+
+```
+scons build/ARM/gem5.opt -j<NUM_CPUS>
+./build/ARM/gem5.opt configs/example/gem5_library/arm-ubuntu-boot-exit.py
+```
+
+"""
+
+from gem5.isas import ISA
+from m5.objects import ArmDefaultRelease
+from gem5.utils.requires import requires
+from gem5.resources.resource import Resource
+from gem5.simulate.simulator import Simulator
+from m5.objects import VExpress_GEM5_Foundation
+from gem5.components.boards.arm_board import ArmBoard
+from gem5.components.memory import DualChannelDDR4_2400
+from gem5.components.processors.cpu_types import CPUTypes
+from gem5.components.processors.simple_processor import SimpleProcessor
+
+# This runs a check to ensure the gem5 binary is compiled for ARM.
+
+requires(
+ isa_required=ISA.ARM,
+)
+
+# With ARM, we use simple caches.
+
+from gem5.components.cachehierarchies.classic\
+ .private_l1_private_l2_cache_hierarchy import (
+ PrivateL1PrivateL2CacheHierarchy,
+)
+
+
+# Here we setup the parameters of the l1 and l2 caches.
+
+cache_hierarchy = PrivateL1PrivateL2CacheHierarchy(
+ l1d_size="16kB",
+ l1i_size="16kB",
+ l2_size="256kB",
+)
+
+# Memory: Dual Channel DDR4 2400 DRAM device.
+
+memory = DualChannelDDR4_2400(size = "2GB")
+
+# Here we setup the processor. We use a simple TIMING processor. The config
+# script was also tested with ATOMIC processor.
+
+processor = SimpleProcessor(
+ cpu_type=CPUTypes.TIMING,
+ num_cores=2,
+)
+
+# The ArmBoard requires a `release` to be specified. This adds all the
+# extensions or features to the system. We are setting this to Armv8
+# (ArmDefaultRelease) in this example config script. However, the ArmBoard
+# currently does not support SECURITY extension.
+
+release = ArmDefaultRelease()
+
+# Removing the SECURITY extension.
+
+release.extensions.remove(release.extensions[2])
+
+# The platform sets up the memory ranges of all the on-chip and off-chip
+# devices present on the ARM system.
+
+platform = VExpress_GEM5_Foundation()
+
+# Here we setup the board. The ArmBoard allows for Full-System ARM simulations.
+
+board = ArmBoard(
+ clk_freq = "3GHz",
+ processor = processor,
+ memory = memory,
+ cache_hierarchy = cache_hierarchy,
+ release = release,
+ platform = platform
+)
+
+# Here we set the Full System workload.
+
+# The `set_kernel_disk_workload` function on the ArmBoard accepts an ARM
+# kernel, a disk image, and, path to the bootloader.
+
+board.set_kernel_disk_workload(
+
+ # The ARM kernel will be automatically downloaded to the `~/.cache/gem5`
+ # directory if not already present. The arm-ubuntu-boot-exit was tested
+ # with `vmlinux.arm64`
+
+ kernel = Resource("arm64-linux-kernel-5.4.49"),
+
+ # The ARM ubuntu image will be automatically downloaded to the
+ # `~/.cache/gem5` directory if not already present.
+
+ disk_image = Resource("arm64-ubuntu-18.04-img"),
+
+ # We need to specify the path for the bootloader file.
+
+ bootloader = Resource("arm64-bootloader-foundation"),
+
+ # For the arm64-ubuntu-18.04.img, we need to specify the readfile content
+
+ readfile_contents = "m5 exit"
+)
+
+# We define the system with the aforementioned system defined.
+
+simulator = Simulator(board = board)
+
+# Once the system successfully boots, it encounters an
+# `m5_exit instruction encountered`. We stop the simulation then. When the
+# simulation has ended you may inspect `m5out/board.terminal` to see
+# the stdout.
+
+simulator.run()
diff --git a/configs/example/gem5_library/checkpoints/riscv-hello-restore-checkpoint.py b/configs/example/gem5_library/checkpoints/riscv-hello-restore-checkpoint.py
new file mode 100644
index 0000000..4b06267
--- /dev/null
+++ b/configs/example/gem5_library/checkpoints/riscv-hello-restore-checkpoint.py
@@ -0,0 +1,112 @@
+# Copyright (c) 2022 The Regents of the University of California
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+This gem5 configuation script creates a simple board sharing the same
+structure as the one in
+configs/example/gem5_library/checkpoint/riscv-hello-save-checkpoint.py.
+This script restores the checkpoint generated by the above script, and
+runs the rest of "riscv-hello" binary simulation.
+This configuration serves as an example of restoring a checkpoint.
+
+This is setup is the close to the simplest setup possible using the gem5
+library. It does not contain any kind of caching, IO, or any non-essential
+components.
+
+Usage
+-----
+
+```
+scons build/RISCV/gem5.opt
+./build/RISCV/gem5.opt \
+ configs/example/gem5_library/checkpoint/riscv-hello-restore-checkpoint.py
+```
+"""
+
+from gem5.isas import ISA
+from gem5.utils.requires import requires
+from gem5.resources.resource import Resource
+from gem5.components.memory import SingleChannelDDR3_1600
+from gem5.components.processors.cpu_types import CPUTypes
+from gem5.components.boards.simple_board import SimpleBoard
+from gem5.components.cachehierarchies.classic.no_cache import NoCache
+from gem5.components.processors.simple_processor import SimpleProcessor
+from gem5.simulate.simulator import Simulator
+
+# This check ensures the gem5 binary is compiled to the RISCV ISA target.
+# If not, an exception will be thrown.
+requires(isa_required=ISA.RISCV)
+
+# In this setup we don't have a cache. `NoCache` can be used for such setups.
+cache_hierarchy = NoCache()
+
+# We use a single channel DDR3_1600 memory system
+memory = SingleChannelDDR3_1600(size="32MB")
+
+# We use a simple Timing processor with one core.
+processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.RISCV,
+ num_cores=1)
+
+# The gem5 library simble board which can be used to run simple SE-mode
+# simulations.
+board = SimpleBoard(
+ clk_freq="3GHz",
+ processor=processor,
+ memory=memory,
+ cache_hierarchy=cache_hierarchy,
+)
+
+# Here we set the workload. In this case we want to run a simple "Hello World!"
+# program compiled to the RISCV ISA. The `Resource` class will automatically
+# download the binary from the gem5 Resources cloud bucket if it's not already
+# present.
+board.set_se_binary_workload(
+ # the workload should be the same as the save-checkpoint script
+ Resource("riscv-hello")
+)
+
+# Getting the pre-taken checkpoint from gem5-resources. This checkpoint
+# was taken from running this gem5 configuration script,
+# configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py
+checkpoint_resource = Resource("riscv-hello-example-checkpoint")
+
+# Now we restore the checkpoint by passing the path to the checkpoint to
+# the Simulator object. The checkpoint_path could be a string containing
+# the path to the checkpoint folder. However, here, we use gem5 resources
+# to automatically download the checkpoint folder, and use .get_local_path()
+# to obtain the path to that folder.
+checkpoint_path = checkpoint_resource.get_local_path()
+print("Restore a checkpoint at", checkpoint_path)
+simulator = Simulator(board=board, full_system=False,
+ checkpoint_path=checkpoint_path)
+simulator.run()
+
+print(
+ "Exiting @ tick {} because {}.".format(
+ simulator.get_current_tick(),
+ simulator.get_last_exit_event_cause(),
+ )
+)
diff --git a/configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py b/configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py
new file mode 100644
index 0000000..fd81d45
--- /dev/null
+++ b/configs/example/gem5_library/checkpoints/riscv-hello-save-checkpoint.py
@@ -0,0 +1,108 @@
+# Copyright (c) 2022 The Regents of the University of California
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+This gem5 configuation script creates a simple board to run the first
+10^6 ticks of "riscv-hello" binary simulation and saves a checkpoint.
+This configuration serves as an example of taking a checkpoint.
+
+This is setup is the close to the simplest setup possible using the gem5
+library. It does not contain any kind of caching, IO, or any non-essential
+components.
+
+Usage
+-----
+
+```
+scons build/RISCV/gem5.opt
+./build/RISCV/gem5.opt \
+ configs/example/gem5_library/checkpoint/riscv-hello-save-checkpoint.py
+```
+"""
+
+from gem5.isas import ISA
+from gem5.utils.requires import requires
+from gem5.resources.resource import Resource
+from gem5.components.memory import SingleChannelDDR3_1600
+from gem5.components.processors.cpu_types import CPUTypes
+from gem5.components.boards.simple_board import SimpleBoard
+from gem5.components.cachehierarchies.classic.no_cache import NoCache
+from gem5.components.processors.simple_processor import SimpleProcessor
+from gem5.simulate.simulator import Simulator
+
+# This check ensures the gem5 binary is compiled to the RISCV ISA target.
+# If not, an exception will be thrown.
+requires(isa_required=ISA.RISCV)
+
+# In this setup we don't have a cache. `NoCache` can be used for such setups.
+cache_hierarchy = NoCache()
+
+# We use a single channel DDR3_1600 memory system
+memory = SingleChannelDDR3_1600(size="32MB")
+
+# We use a simple Timing processor with one core.
+processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, isa=ISA.RISCV,
+ num_cores=1)
+
+# The gem5 library simble board which can be used to run simple SE-mode
+# simulations.
+board = SimpleBoard(
+ clk_freq="3GHz",
+ processor=processor,
+ memory=memory,
+ cache_hierarchy=cache_hierarchy,
+)
+
+# Here we set the workload. In this case we want to run a simple "Hello World!"
+# program compiled to the RISCV ISA. The `Resource` class will automatically
+# download the binary from the gem5 Resources cloud bucket if it's not already
+# present.
+board.set_se_binary_workload(
+ # The `Resource` class reads the `resources.json` file from the gem5
+ # resources repository:
+ # https://gem5.googlesource.com/public/gem5-resource.
+ # Any resource specified in this file will be automatically retrieved.
+ # At the time of writing, this file is a WIP and does not contain all
+ # resources. Jira ticket: https://gem5.atlassian.net/browse/GEM5-1096
+ Resource("riscv-hello")
+)
+
+# Lastly we run the simulation.
+max_ticks = 10**6
+simulator = Simulator(board=board, full_system=False)
+simulator.run(max_ticks = max_ticks)
+
+print(
+ "Exiting @ tick {} because {}.".format(
+ simulator.get_current_tick(),
+ simulator.get_last_exit_event_cause(),
+ )
+)
+
+checkpoint_path = "riscv-hello-checkpoint/"
+print("Taking a checkpoint at", checkpoint_path)
+simulator.save_checkpoint(checkpoint_path)
+print("Done taking a checkpoint")
diff --git a/configs/example/gem5_library/riscv-fs.py b/configs/example/gem5_library/riscv-fs.py
index 4c1f117..dffb3d4 100644
--- a/configs/example/gem5_library/riscv-fs.py
+++ b/configs/example/gem5_library/riscv-fs.py
@@ -66,7 +66,9 @@
memory = SingleChannelDDR3_1600()
# Setup a single core Processor.
-processor = SimpleProcessor(cpu_type=CPUTypes.TIMING, num_cores=1)
+processor = SimpleProcessor(
+ cpu_type=CPUTypes.TIMING, isa=ISA.RISCV, num_cores=1
+)
# Setup the board.
board = RiscvBoard(
diff --git a/configs/example/gem5_library/riscv-ubuntu-run.py b/configs/example/gem5_library/riscv-ubuntu-run.py
index f25d20f..f3e6d13 100644
--- a/configs/example/gem5_library/riscv-ubuntu-run.py
+++ b/configs/example/gem5_library/riscv-ubuntu-run.py
@@ -53,6 +53,7 @@
from gem5.isas import ISA
from gem5.coherence_protocol import CoherenceProtocol
from gem5.resources.resource import Resource
+from gem5.simulate.simulator import Simulator
# This runs a check to ensure the gem5 binary is compiled for RISCV.
@@ -79,6 +80,7 @@
# Here we setup the processor. We use a simple processor.
processor = SimpleProcessor(
cpu_type=CPUTypes.TIMING,
+ isa=ISA.RISCV,
num_cores=2,
)
@@ -113,34 +115,5 @@
),
)
-root = Root(full_system=True, system=board)
-
-m5.instantiate()
-
-# We simulate the system till we encounter `m5_exit instruction encountered`.
-
-exit_event = m5.simulate()
-
-# We check whether the simulation ended with `m5_exit instruction encountered`
-
-if exit_event.getCause() == "m5_exit instruction encountered":
- # We acknowledge the user that the boot was successful.
-
- print("Successfully completed booting!")
-else:
- # `m5_exit instruction encountered` was never encountered. We exit the
- # program unsuccessfully.
-
- print("The startup was not completed successfully!",)
- print(
- "Exiting @ tick {} because {}."\
- .format(m5.curTick(), exit_event.getCause())
- )
- exit(-1)
-
-# We are done with the simulation. We exit the program now.
-
-print(
-"Exiting @ tick {} because {}."\
- .format(m5.curTick(), exit_event.getCause())
-)
+simulator = Simulator(board=board)
+simulator.run()
diff --git a/configs/example/gem5_library/x86-gapbs-benchmarks.py b/configs/example/gem5_library/x86-gapbs-benchmarks.py
index 92746aa..50f56d5 100644
--- a/configs/example/gem5_library/x86-gapbs-benchmarks.py
+++ b/configs/example/gem5_library/x86-gapbs-benchmarks.py
@@ -147,6 +147,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gem5_library/x86-npb-benchmarks.py b/configs/example/gem5_library/x86-npb-benchmarks.py
index 1609521..83cc700 100644
--- a/configs/example/gem5_library/x86-npb-benchmarks.py
+++ b/configs/example/gem5_library/x86-npb-benchmarks.py
@@ -164,6 +164,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gem5_library/x86-parsec-benchmarks.py b/configs/example/gem5_library/x86-parsec-benchmarks.py
index c3dd9ea..0d2e665 100644
--- a/configs/example/gem5_library/x86-parsec-benchmarks.py
+++ b/configs/example/gem5_library/x86-parsec-benchmarks.py
@@ -137,6 +137,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gem5_library/x86-spec-cpu2006-benchmarks.py b/configs/example/gem5_library/x86-spec-cpu2006-benchmarks.py
index c1b7a3b..2e52eef 100644
--- a/configs/example/gem5_library/x86-spec-cpu2006-benchmarks.py
+++ b/configs/example/gem5_library/x86-spec-cpu2006-benchmarks.py
@@ -187,6 +187,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gem5_library/x86-spec-cpu2017-benchmarks.py b/configs/example/gem5_library/x86-spec-cpu2017-benchmarks.py
index 2a03389..4e77dd0 100644
--- a/configs/example/gem5_library/x86-spec-cpu2017-benchmarks.py
+++ b/configs/example/gem5_library/x86-spec-cpu2017-benchmarks.py
@@ -193,6 +193,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gem5_library/x86-ubuntu-run-with-kvm.py b/configs/example/gem5_library/x86-ubuntu-run-with-kvm.py
index 1aea258..a47eb2b 100644
--- a/configs/example/gem5_library/x86-ubuntu-run-with-kvm.py
+++ b/configs/example/gem5_library/x86-ubuntu-run-with-kvm.py
@@ -89,6 +89,7 @@
processor = SimpleSwitchableProcessor(
starting_core_type=CPUTypes.KVM,
switch_core_type=CPUTypes.TIMING,
+ isa=ISA.X86,
num_cores=2,
)
diff --git a/configs/example/gpufs/DisjointNetwork.py b/configs/example/gpufs/DisjointNetwork.py
new file mode 100644
index 0000000..e1838bb
--- /dev/null
+++ b/configs/example/gpufs/DisjointNetwork.py
@@ -0,0 +1,107 @@
+# Copyright (c) 2021 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+from m5.objects import *
+from m5.util import fatal
+
+from importlib import *
+
+from network import Network
+
+class DisjointSimple(SimpleNetwork):
+
+ def __init__(self, ruby_system):
+ super(DisjointSimple, self).__init__()
+
+ self.netifs = []
+ self.routers = []
+ self.int_links = []
+ self.ext_links = []
+ self.ruby_system = ruby_system
+
+ def connectCPU(self, opts, controllers):
+
+ # Setup parameters for makeTopology call for CPU network
+ topo_module = import_module("topologies.%s" % opts.cpu_topology)
+ topo_class = getattr(topo_module, opts.cpu_topology)
+ _topo = topo_class(controllers)
+ _topo.makeTopology(opts, self, SimpleIntLink,
+ SimpleExtLink, Switch)
+
+ self.initSimple(opts, self.int_links, self.ext_links)
+
+ def connectGPU(self, opts, controllers):
+
+ # Setup parameters for makeTopology call for GPU network
+ topo_module = import_module("topologies.%s" % opts.gpu_topology)
+ topo_class = getattr(topo_module, opts.gpu_topology)
+ _topo = topo_class(controllers)
+ _topo.makeTopology(opts, self, SimpleIntLink,
+ SimpleExtLink, Switch)
+
+ self.initSimple(opts, self.int_links, self.ext_links)
+
+
+ def initSimple(self, opts, int_links, ext_links):
+
+ # Attach links to network
+ self.int_links = int_links
+ self.ext_links = ext_links
+
+ self.setup_buffers()
+
+class DisjointGarnet(GarnetNetwork):
+
+ def __init__(self, ruby_system):
+ super(DisjointGarnet, self).__init__()
+
+ self.netifs = []
+ self.ruby_system = ruby_system
+
+ def connectCPU(self, opts, controllers):
+
+ # Setup parameters for makeTopology call for CPU network
+ topo_module = import_module("topologies.%s" % opts.cpu_topology)
+ topo_class = getattr(topo_module, opts.cpu_topology)
+ _topo = topo_class(controllers)
+ _topo.makeTopology(opts, self, GarnetIntLink,
+ GarnetExtLink, GarnetRouter)
+
+ Network.init_network(opts, self, GarnetNetworkInterface)
+
+ def connectGPU(self, opts, controllers):
+
+ # Setup parameters for makeTopology call
+ topo_module = import_module("topologies.%s" % opts.gpu_topology)
+ topo_class = getattr(topo_module, opts.gpu_topology)
+ _topo = topo_class(controllers)
+ _topo.makeTopology(opts, self, GarnetIntLink,
+ GarnetExtLink, GarnetRouter)
+
+ Network.init_network(opts, self, GarnetNetworkInterface)
diff --git a/configs/example/gpufs/Disjoint_VIPER.py b/configs/example/gpufs/Disjoint_VIPER.py
new file mode 100644
index 0000000..8ddaeac
--- /dev/null
+++ b/configs/example/gpufs/Disjoint_VIPER.py
@@ -0,0 +1,196 @@
+# Copyright (c) 2021 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+from m5.defines import buildEnv
+from m5.objects import *
+from m5.util import fatal
+
+from example.gpufs.DisjointNetwork import *
+from ruby.GPU_VIPER import *
+from ruby import Ruby
+
+
+class DummySystem():
+ def __init__(self, mem_ranges):
+
+ self.mem_ctrls = []
+ self.mem_ranges = mem_ranges
+
+
+class Disjoint_VIPER(RubySystem):
+ def __init__(self):
+ if buildEnv['PROTOCOL'] != "GPU_VIPER":
+ fatal("This ruby config only supports the GPU_VIPER protocol")
+
+ super(Disjoint_VIPER, self).__init__()
+
+ def create(self, options, system, piobus, dma_devices):
+
+ # Disjoint network topology
+ if "garnet" in options.network:
+ self.network_cpu = DisjointGarnet(self)
+ self.network_gpu = DisjointGarnet(self)
+ else:
+ self.network_cpu = DisjointSimple(self)
+ self.network_gpu = DisjointSimple(self)
+
+
+ # Construct CPU controllers
+ cpu_dir_nodes = \
+ construct_dirs(options, system, self, self.network_cpu)
+ (cp_sequencers, cp_cntrl_nodes) = \
+ construct_corepairs(options, system, self, self.network_cpu)
+
+ # Construct GPU controllers
+ (tcp_sequencers, tcp_cntrl_nodes) = \
+ construct_tcps(options, system, self, self.network_gpu)
+ (sqc_sequencers, sqc_cntrl_nodes) = \
+ construct_sqcs(options, system, self, self.network_gpu)
+ (scalar_sequencers, scalar_cntrl_nodes) = \
+ construct_scalars(options, system, self, self.network_gpu)
+ tcc_cntrl_nodes = \
+ construct_tccs(options, system, self, self.network_gpu)
+
+ # Construct CPU memories
+ Ruby.setup_memory_controllers(system, self, cpu_dir_nodes, options)
+
+ # Construct GPU memories
+ (gpu_dir_nodes, gpu_mem_ctrls) = \
+ construct_gpudirs(options, system, self, self.network_gpu)
+
+ # Configure the directories based on which network they are in
+ for cpu_dir_node in cpu_dir_nodes:
+ cpu_dir_node.CPUonly = True
+ cpu_dir_node.GPUonly = False
+ for gpu_dir_node in gpu_dir_nodes:
+ gpu_dir_node.CPUonly = False
+ gpu_dir_node.GPUonly = True
+
+ # Set access backing store if specified
+ if options.access_backing_store:
+ self.access_backing_store = True
+
+ # Assign the memory controllers to the system
+ cpu_abstract_mems = []
+ for mem_ctrl in system.mem_ctrls:
+ cpu_abstract_mems.append(mem_ctrl.dram)
+ system.memories = cpu_abstract_mems
+
+ gpu_abstract_mems = []
+ for mem_ctrl in gpu_mem_ctrls:
+ gpu_abstract_mems.append(mem_ctrl.dram)
+ system.pc.south_bridge.gpu.memories = gpu_abstract_mems
+
+ # Setup DMA controllers
+ gpu_dma_types = ["VegaPagetableWalker", "AMDGPUMemoryManager"]
+
+ cpu_dma_ctrls = []
+ gpu_dma_ctrls = []
+ dma_cntrls = []
+ for i, dma_device in enumerate(dma_devices):
+ dma_seq = DMASequencer(version=i, ruby_system=self)
+ dma_cntrl = DMA_Controller(version=i, dma_sequencer=dma_seq,
+ ruby_system=self)
+
+ # Handle inconsistently named ports on various DMA devices:
+ if not hasattr(dma_device, 'type'):
+ # IDE doesn't have a .type but seems like everything else does.
+ dma_seq.in_ports = dma_device
+ elif dma_device.type in gpu_dma_types:
+ dma_seq.in_ports = dma_device.port
+ else:
+ dma_seq.in_ports = dma_device.dma
+
+ if hasattr(dma_device, 'type') and \
+ dma_device.type in gpu_dma_types:
+ dma_cntrl.requestToDir = MessageBuffer(buffer_size=0)
+ dma_cntrl.requestToDir.out_port = self.network_gpu.in_port
+ dma_cntrl.responseFromDir = MessageBuffer(buffer_size=0)
+ dma_cntrl.responseFromDir.in_port = self.network_gpu.out_port
+ dma_cntrl.mandatoryQueue = MessageBuffer(buffer_size = 0)
+
+ gpu_dma_ctrls.append(dma_cntrl)
+ else:
+ dma_cntrl.requestToDir = MessageBuffer(buffer_size=0)
+ dma_cntrl.requestToDir.out_port = self.network_cpu.in_port
+ dma_cntrl.responseFromDir = MessageBuffer(buffer_size=0)
+ dma_cntrl.responseFromDir.in_port = self.network_cpu.out_port
+ dma_cntrl.mandatoryQueue = MessageBuffer(buffer_size = 0)
+
+ cpu_dma_ctrls.append(dma_cntrl)
+
+ dma_cntrls.append(dma_cntrl)
+
+ system.dma_cntrls = dma_cntrls
+
+
+ # Collect CPU and GPU controllers into seperate lists
+ cpu_cntrls = cpu_dir_nodes + cp_cntrl_nodes + cpu_dma_ctrls
+ gpu_cntrls = tcp_cntrl_nodes + sqc_cntrl_nodes + \
+ scalar_cntrl_nodes + tcc_cntrl_nodes + gpu_dma_ctrls + \
+ gpu_dir_nodes
+
+
+ # Setup number of vnets
+ self.number_of_virtual_networks = 11
+ self.network_cpu.number_of_virtual_networks = 11
+ self.network_gpu.number_of_virtual_networks = 11
+
+
+ # Set up the disjoint topology
+ self.network_cpu.connectCPU(options, cpu_cntrls)
+ self.network_gpu.connectGPU(options, gpu_cntrls)
+
+
+ # Create port proxy for connecting system port. System port is used
+ # for loading from outside guest, e.g., binaries like vmlinux.
+ system.sys_port_proxy = RubyPortProxy(ruby_system = self)
+ system.sys_port_proxy.pio_request_port = piobus.cpu_side_ports
+ system.system_port = system.sys_port_proxy.in_ports
+
+
+ # Only CPU sequencers connect to PIO bus. This acts as the "default"
+ # destination for unknown address ranges. PCIe requests fall under
+ # this category.
+ for i in range(len(cp_sequencers)):
+ cp_sequencers[i].pio_request_port = piobus.cpu_side_ports
+ cp_sequencers[i].mem_request_port = piobus.cpu_side_ports
+
+ # The CorePairs in MOESI_AMD_Base round up when constructing
+ # sequencers, but if the CPU does not exit there would be no
+ # sequencer to send a range change, leading to assert.
+ if i < options.num_cpus:
+ cp_sequencers[i].pio_response_port = piobus.mem_side_ports
+
+
+ # Setup ruby port. Both CPU and GPU are actually connected here.
+ all_sequencers = cp_sequencers + tcp_sequencers + \
+ sqc_sequencers + scalar_sequencers
+ self._cpu_ports = all_sequencers
+ self.num_of_sequencers = len(all_sequencers)
diff --git a/configs/example/gpufs/hip_cookbook.py b/configs/example/gpufs/hip_cookbook.py
new file mode 100644
index 0000000..cd0e284
--- /dev/null
+++ b/configs/example/gpufs/hip_cookbook.py
@@ -0,0 +1,131 @@
+# Copyright (c) 2022 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+import m5
+import runfs
+import tempfile
+import argparse
+import sys
+import os
+
+from amd import AmdGPUOptions
+from common import Options
+from common import GPUTLBOptions
+from ruby import Ruby
+
+cookbook_runscript = '''\
+export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
+export HSA_ENABLE_SDMA=0
+dmesg -n3
+dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
+if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
+ echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
+ /sbin/m5 exit
+fi
+modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
+echo "Running {}"
+cd /opt/rocm/hip/samples/2_Cookbook/{}/
+make clean
+make
+/sbin/m5 exit
+'''
+
+def addCookbookOptions(parser):
+ parser.add_argument("-a", "--app", default=None,
+ choices=['0_MatrixTranspose',
+ '1_hipEvent',
+ '3_shared_memory',
+ '4_shfl',
+ '5_2dshfl',
+ '6_dynamic_shared',
+ '7_streams',
+ '8_peer2peer',
+ '9_unroll',
+ '10_inline_asm',
+ '11_texture_driver',
+ '13_occupancy',
+ '14_gpu_arch',
+ '15_static_library'],
+ help="GPU application to run")
+ parser.add_argument("-o", "--opts", default="",
+ help="GPU application arguments")
+
+if __name__ == "__m5_main__":
+ parser = argparse.ArgumentParser()
+ runfs.addRunFSOptions(parser)
+ Options.addCommonOptions(parser)
+ AmdGPUOptions.addAmdGPUOptions(parser)
+ Ruby.define_options(parser)
+ GPUTLBOptions.tlb_options(parser)
+ addCookbookOptions(parser)
+
+ # Parse now so we can override options
+ args = parser.parse_args()
+
+ # Create temp script to run application
+ if args.app is None:
+ print("No application given. Use %s -a <app>" % sys.argv[0])
+ sys.exit(1)
+ elif args.kernel is None:
+ print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.disk_image is None:
+ print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.gpu_mmio_trace is None:
+ print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
+ % sys.argv[0])
+ sys.exit(1)
+
+ _, tempRunscript = tempfile.mkstemp()
+ with open(tempRunscript, 'w') as b64file:
+ runscriptStr = cookbook_runscript.format(args.app, args.app)
+ b64file.write(runscriptStr)
+
+ if args.second_disk == None:
+ args.second_disk = args.disk_image
+
+ # Defaults for Vega10
+ args.ruby = True
+ args.cpu_type = 'X86KvmCPU'
+ args.num_cpus = 1
+ args.mem_size = '3GB'
+ args.dgpu = True
+ args.dgpu_mem_size = '16GB'
+ args.dgpu_start = '0GB'
+ args.checkpoint_restore = 0
+ args.disjoint = True
+ args.timing_gpu = True
+ args.script = tempRunscript
+ args.dgpu_xor_low_bit = 0
+
+ print(args.disk_image)
+
+ # Run gem5
+ runfs.runGpuFSSystem(args)
diff --git a/configs/example/gpufs/hip_rodinia.py b/configs/example/gpufs/hip_rodinia.py
new file mode 100644
index 0000000..3d7cef4
--- /dev/null
+++ b/configs/example/gpufs/hip_rodinia.py
@@ -0,0 +1,139 @@
+# Copyright (c) 2022 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+import m5
+import runfs
+import base64
+import tempfile
+import argparse
+import sys
+import os
+
+from amd import AmdGPUOptions
+from common import Options
+from common import GPUTLBOptions
+from ruby import Ruby
+
+rodinia_runscript = '''\
+export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
+export HSA_ENABLE_SDMA=0
+dmesg -n3
+dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
+if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
+ echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
+ /sbin/m5 exit
+fi
+modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
+echo "Running {}"
+cd /home/gem5/HIP-Examples/rodinia_3.0/hip/{}/
+make clean
+make
+make test
+/sbin/m5 exit
+'''
+
+def addRodiniaOptions(parser):
+ parser.add_argument("-a", "--app", default=None,
+ choices=['b+tree',
+ 'backprop',
+ 'bfs',
+ 'cfd',
+ 'dwt2d',
+ 'gaussian',
+ 'heartwall',
+ 'hotspot',
+ 'hybridsort',
+ 'kmeans',
+ 'lavaMD',
+ 'leukocyte',
+ 'lud',
+ 'myocyte',
+ 'nn',
+ 'nw',
+ 'particlefilter',
+ 'pathfinder',
+ 'srad',
+ 'streamcluster'],
+ help="GPU application to run")
+ parser.add_argument("-o", "--opts", default="",
+ help="GPU application arguments")
+
+if __name__ == "__m5_main__":
+ parser = argparse.ArgumentParser()
+ runfs.addRunFSOptions(parser)
+ Options.addCommonOptions(parser)
+ AmdGPUOptions.addAmdGPUOptions(parser)
+ Ruby.define_options(parser)
+ GPUTLBOptions.tlb_options(parser)
+ addRodiniaOptions(parser)
+
+ # Parse now so we can override options
+ args = parser.parse_args()
+
+ # Create temp script to run application
+ if args.app is None:
+ print("No application given. Use %s -a <app>" % sys.argv[0])
+ sys.exit(1)
+ elif args.kernel is None:
+ print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.disk_image is None:
+ print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.gpu_mmio_trace is None:
+ print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
+ % sys.argv[0])
+ sys.exit(1)
+
+ _, tempRunscript = tempfile.mkstemp()
+ with open(tempRunscript, 'w') as b64file:
+ runscriptStr = rodinia_runscript.format(args.app, args.app)
+ b64file.write(runscriptStr)
+
+ if args.second_disk == None:
+ args.second_disk = args.disk_image
+
+ # Defaults for Vega10
+ args.ruby = True
+ args.cpu_type = 'X86KvmCPU'
+ args.num_cpus = 1
+ args.mem_size = '3GB'
+ args.dgpu = True
+ args.dgpu_mem_size = '16GB'
+ args.dgpu_start = '0GB'
+ args.checkpoint_restore = 0
+ args.disjoint = True
+ args.timing_gpu = True
+ args.script = tempRunscript
+ args.dgpu_xor_low_bit = 0
+
+ print(args.disk_image)
+
+ # Run gem5
+ runfs.runGpuFSSystem(args)
diff --git a/configs/example/gpufs/hip_samples.py b/configs/example/gpufs/hip_samples.py
new file mode 100644
index 0000000..b3a66a7
--- /dev/null
+++ b/configs/example/gpufs/hip_samples.py
@@ -0,0 +1,129 @@
+# Copyright (c) 2022 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+import m5
+import runfs
+import tempfile
+import argparse
+import sys
+import os
+
+from amd import AmdGPUOptions
+from common import Options
+from common import GPUTLBOptions
+from ruby import Ruby
+
+samples_runscript = '''\
+export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
+export HSA_ENABLE_SDMA=0
+dmesg -n3
+dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
+if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
+ echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
+ /sbin/m5 exit
+fi
+modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
+echo "Running {}"
+cd /home/gem5/HIP-Examples/HIP-Examples-Applications/{}/
+make clean
+make
+/sbin/m5 exit
+'''
+
+def addSamplesOptions(parser):
+ parser.add_argument("-a", "--app", default=None,
+ choices=['BinomialOption',
+ 'BitonicSort',
+ 'FastWalshTransform',
+ 'FloydWarshall',
+ 'HelloWorld',
+ 'Histogram',
+ 'MatrixMultiplication',
+ 'PrefixSum',
+ 'RecursiveGaussian',
+ 'SimpleConvolution',
+ 'dct',
+ 'dwtHaar1D'],
+ help="GPU application to run")
+ parser.add_argument("-o", "--opts", default="",
+ help="GPU application arguments")
+
+if __name__ == "__m5_main__":
+ parser = argparse.ArgumentParser()
+ runfs.addRunFSOptions(parser)
+ Options.addCommonOptions(parser)
+ AmdGPUOptions.addAmdGPUOptions(parser)
+ Ruby.define_options(parser)
+ GPUTLBOptions.tlb_options(parser)
+ addSamplesOptions(parser)
+
+ # Parse now so we can override options
+ args = parser.parse_args()
+
+ # Create temp script to run application
+ if args.app is None:
+ print("No application given. Use %s -a <app>" % sys.argv[0])
+ sys.exit(1)
+ elif args.kernel is None:
+ print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.disk_image is None:
+ print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.gpu_mmio_trace is None:
+ print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
+ % sys.argv[0])
+ sys.exit(1)
+
+ _, tempRunscript = tempfile.mkstemp()
+ with open(tempRunscript, 'w') as b64file:
+ runscriptStr = samples_runscript.format(args.app, args.app)
+ b64file.write(runscriptStr)
+
+ if args.second_disk == None:
+ args.second_disk = args.disk_image
+
+ # Defaults for Vega10
+ args.ruby = True
+ args.cpu_type = 'X86KvmCPU'
+ args.num_cpus = 1
+ args.mem_size = '3GB'
+ args.dgpu = True
+ args.dgpu_mem_size = '16GB'
+ args.dgpu_start = '0GB'
+ args.checkpoint_restore = 0
+ args.disjoint = True
+ args.timing_gpu = True
+ args.script = tempRunscript
+ args.dgpu_xor_low_bit = 0
+
+ print(args.disk_image)
+
+ # Run gem5
+ runfs.runGpuFSSystem(args)
diff --git a/configs/example/gpufs/runfs.py b/configs/example/gpufs/runfs.py
index e5acf05..b198552 100644
--- a/configs/example/gpufs/runfs.py
+++ b/configs/example/gpufs/runfs.py
@@ -71,7 +71,20 @@
help="Take a checkpoint before driver sends MMIOs. "
"This is used to switch out of KVM mode and into "
"timing mode required to read the VGA ROM on boot.")
-
+ parser.add_argument("--cpu-topology", type=str, default="Crossbar",
+ help="Network topology to use for CPU side. "
+ "Check configs/topologies for complete set")
+ parser.add_argument("--gpu-topology", type=str, default="Crossbar",
+ help="Network topology to use for GPU side. "
+ "Check configs/topologies for complete set")
+ parser.add_argument("--dgpu-mem-size", action="store", type=str,
+ default="16GB", help="Specify the dGPU physical memory"
+ " size")
+ parser.add_argument("--dgpu-num-dirs", type=int, default=1, help="Set "
+ "the number of dGPU directories (memory controllers")
+ parser.add_argument("--dgpu-mem-type", default="HBM_1000_4H_1x128",
+ choices=ObjectList.mem_list.get_names(),
+ help="type of memory to use")
def runGpuFSSystem(args):
'''
diff --git a/configs/example/gpufs/system/system.py b/configs/example/gpufs/system/system.py
index d06cd2c..972a4f9 100644
--- a/configs/example/gpufs/system/system.py
+++ b/configs/example/gpufs/system/system.py
@@ -33,9 +33,12 @@
from common.Benchmarks import *
from common.FSConfig import *
+from common import GPUTLBConfig
from common import Simulation
from ruby import Ruby
+from example.gpufs.Disjoint_VIPER import *
+
def makeGpuFSSystem(args):
# Boot options are standard gem5 options plus:
# - Framebuffer device emulation 0 to reduce driver code paths.
@@ -54,7 +57,10 @@
# Use the common FSConfig to setup a Linux X86 System
(TestCPUClass, test_mem_mode, FutureClass) = Simulation.setCPUClass(args)
- bm = SysConfig(disks=[args.disk_image], mem=args.mem_size)
+ disks = [args.disk_image]
+ if args.second_disk is not None:
+ disks.extend([args.second_disk])
+ bm = SysConfig(disks=disks, mem=args.mem_size)
system = makeLinuxX86System(test_mem_mode, args.num_cpus, bm, True,
cmdline=cmdline)
system.workload.object_file = binary(args.kernel)
@@ -77,45 +83,102 @@
system.shadow_rom_ranges = [AddrRange(0xc0000, size = Addr('128kB'))]
# Create specified number of CPUs. GPUFS really only needs one.
- system.cpu = [TestCPUClass(clk_domain=system.cpu_clk_domain, cpu_id=i)
+ system.cpu = [X86KvmCPU(clk_domain=system.cpu_clk_domain, cpu_id=i)
for i in range(args.num_cpus)]
-
- if ObjectList.is_kvm_cpu(TestCPUClass) or \
- ObjectList.is_kvm_cpu(FutureClass):
- system.kvm_vm = KvmVM()
+ system.kvm_vm = KvmVM()
# Create AMDGPU and attach to southbridge
shader = createGPU(system, args)
connectGPU(system, args)
+ # The shader core will be whatever is after the CPU cores are accounted for
+ shader_idx = args.num_cpus
+ system.cpu.append(shader)
+
# This arbitrary address is something in the X86 I/O hole
hsapp_gpu_map_paddr = 0xe00000000
+ hsapp_pt_walker = VegaPagetableWalker()
gpu_hsapp = HSAPacketProcessor(pioAddr=hsapp_gpu_map_paddr,
- numHWQueues=args.num_hw_queues)
+ numHWQueues=args.num_hw_queues,
+ walker=hsapp_pt_walker)
dispatcher = GPUDispatcher()
+ cp_pt_walker = VegaPagetableWalker()
gpu_cmd_proc = GPUCommandProcessor(hsapp=gpu_hsapp,
- dispatcher=dispatcher)
+ dispatcher=dispatcher,
+ walker=cp_pt_walker)
shader.dispatcher = dispatcher
shader.gpu_cmd_proc = gpu_cmd_proc
+ system.pc.south_bridge.gpu.cp = gpu_cmd_proc
+
+ # GPU Interrupt Handler
+ device_ih = AMDGPUInterruptHandler()
+ system.pc.south_bridge.gpu.device_ih = device_ih
+
+ # Setup the SDMA engines
+ sdma0_pt_walker = VegaPagetableWalker()
+ sdma1_pt_walker = VegaPagetableWalker()
+
+ sdma0 = SDMAEngine(walker=sdma0_pt_walker)
+ sdma1 = SDMAEngine(walker=sdma1_pt_walker)
+
+ system.pc.south_bridge.gpu.sdma0 = sdma0
+ system.pc.south_bridge.gpu.sdma1 = sdma1
+
+ # Setup PM4 packet processor
+ pm4_pkt_proc = PM4PacketProcessor()
+ system.pc.south_bridge.gpu.pm4_pkt_proc = pm4_pkt_proc
+
+ # GPU data path
+ gpu_mem_mgr = AMDGPUMemoryManager()
+ system.pc.south_bridge.gpu.memory_manager = gpu_mem_mgr
+
+ # CPU data path (SystemHub)
+ system_hub = AMDGPUSystemHub()
+ shader.system_hub = system_hub
+
# GPU, HSAPP, and GPUCommandProc are DMA devices
system._dma_ports.append(gpu_hsapp)
system._dma_ports.append(gpu_cmd_proc)
system._dma_ports.append(system.pc.south_bridge.gpu)
+ system._dma_ports.append(sdma0)
+ system._dma_ports.append(sdma1)
+ system._dma_ports.append(device_ih)
+ system._dma_ports.append(pm4_pkt_proc)
+ system._dma_ports.append(system_hub)
+ system._dma_ports.append(gpu_mem_mgr)
+ system._dma_ports.append(hsapp_pt_walker)
+ system._dma_ports.append(cp_pt_walker)
+ system._dma_ports.append(sdma0_pt_walker)
+ system._dma_ports.append(sdma1_pt_walker)
gpu_hsapp.pio = system.iobus.mem_side_ports
gpu_cmd_proc.pio = system.iobus.mem_side_ports
system.pc.south_bridge.gpu.pio = system.iobus.mem_side_ports
+ sdma0.pio = system.iobus.mem_side_ports
+ sdma1.pio = system.iobus.mem_side_ports
+ device_ih.pio = system.iobus.mem_side_ports
+ pm4_pkt_proc.pio = system.iobus.mem_side_ports
+ system_hub.pio = system.iobus.mem_side_ports
- # Create Ruby system using Ruby.py for now
- Ruby.create_system(args, True, system, system.iobus,
- system._dma_ports)
+ # Full system needs special TLBs for SQC, Scalar, and vector data ports
+ args.full_system = True
+ GPUTLBConfig.config_tlb_hierarchy(args, system, shader_idx,
+ system.pc.south_bridge.gpu, True)
+
+ # Create Ruby system using disjoint VIPER topology
+ system.ruby = Disjoint_VIPER()
+ system.ruby.create(args, system, system.iobus, system._dma_ports)
# Create a seperate clock domain for Ruby
system.ruby.clk_domain = SrcClockDomain(clock = args.ruby_clock,
voltage_domain = system.voltage_domain)
for (i, cpu) in enumerate(system.cpu):
+ # Break once we reach the shader "CPU"
+ if i == args.num_cpus:
+ break
+
#
# Tie the cpu ports to the correct ruby system ports
#
@@ -125,9 +188,8 @@
system.ruby._cpu_ports[i].connectCpuPorts(cpu)
- # The shader core will be whatever is after the CPU cores are accounted for
- shader_idx = args.num_cpus
- system.cpu.append(shader)
+ for j in range(len(system.cpu[i].isa)):
+ system.cpu[i].isa[j].vendor_string = "AuthenticAMD"
gpu_port_idx = len(system.ruby._cpu_ports) \
- args.num_compute_units - args.num_sqc \
diff --git a/configs/example/gpufs/vega10_kvm.py b/configs/example/gpufs/vega10_kvm.py
new file mode 100644
index 0000000..baee077
--- /dev/null
+++ b/configs/example/gpufs/vega10_kvm.py
@@ -0,0 +1,124 @@
+# Copyright (c) 2022 Advanced Micro Devices, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its
+# contributors may be used to endorse or promote products derived from this
+# software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+
+import m5
+import runfs
+import base64
+import tempfile
+import argparse
+import sys
+import os
+
+from amd import AmdGPUOptions
+from common import Options
+from common import GPUTLBOptions
+from ruby import Ruby
+
+
+demo_runscript = '''\
+export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
+export HSA_ENABLE_SDMA=0
+dmesg -n3
+dd if=/root/roms/vega10.rom of=/dev/mem bs=1k seek=768 count=128
+if [ ! -f /lib/modules/`uname -r`/updates/dkms/amdgpu.ko ]; then
+ echo "ERROR: Missing DKMS package for kernel `uname -r`. Exiting gem5."
+ /sbin/m5 exit
+fi
+modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
+echo "Running {} {}"
+echo "{}" | base64 -d > myapp
+chmod +x myapp
+./myapp {}
+/sbin/m5 exit
+'''
+
+def addDemoOptions(parser):
+ parser.add_argument("-a", "--app", default=None,
+ help="GPU application to run")
+ parser.add_argument("-o", "--opts", default="",
+ help="GPU application arguments")
+
+if __name__ == "__m5_main__":
+ parser = argparse.ArgumentParser()
+ runfs.addRunFSOptions(parser)
+ Options.addCommonOptions(parser)
+ AmdGPUOptions.addAmdGPUOptions(parser)
+ Ruby.define_options(parser)
+ GPUTLBOptions.tlb_options(parser)
+ addDemoOptions(parser)
+
+ # Parse now so we can override options
+ args = parser.parse_args()
+
+ # Create temp script to run application
+ if args.app is None:
+ print("No application given. Use %s -a <app>" % sys.argv[0])
+ sys.exit(1)
+ elif args.kernel is None:
+ print("No kernel path given. Use %s --kernel <vmlinux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.disk_image is None:
+ print("No disk path given. Use %s --disk-image <linux>" % sys.argv[0])
+ sys.exit(1)
+ elif args.gpu_mmio_trace is None:
+ print("No MMIO trace path. Use %s --gpu-mmio-trace <path>"
+ % sys.argv[0])
+ sys.exit(1)
+ elif not os.path.isfile(args.app):
+ print("Could not find applcation", args.app)
+ sys.exit(1)
+
+ with open(os.path.abspath(args.app), 'rb') as binfile:
+ encodedBin = base64.b64encode(binfile.read()).decode()
+
+ _, tempRunscript = tempfile.mkstemp()
+ with open(tempRunscript, 'w') as b64file:
+ runscriptStr = demo_runscript.format(args.app, args.opts, encodedBin,
+ args.opts)
+ b64file.write(runscriptStr)
+
+ if args.second_disk == None:
+ args.second_disk = args.disk_image
+
+ # Defaults for Vega10
+ args.ruby = True
+ args.cpu_type = 'X86KvmCPU'
+ args.num_cpus = 1
+ args.mem_size = '3GB'
+ args.dgpu = True
+ args.dgpu_mem_size = '16GB'
+ args.dgpu_start = '0GB'
+ args.checkpoint_restore = 0
+ args.disjoint = True
+ args.timing_gpu = True
+ args.script = tempRunscript
+ args.dgpu_xor_low_bit = 0
+
+ # Run gem5
+ runfs.runGpuFSSystem(args)
diff --git a/configs/example/hmc_hello.py b/configs/example/hmc_hello.py
index 4507768..11d52c0 100644
--- a/configs/example/hmc_hello.py
+++ b/configs/example/hmc_hello.py
@@ -49,7 +49,7 @@
system = System()
# use timing mode for the interaction between requestor-responder ports
system.mem_mode = 'timing'
-# set the clock fequency of the system
+# set the clock frequency of the system
clk = '1GHz'
vd = VoltageDomain(voltage='1V')
system.clk_domain = SrcClockDomain(clock=clk, voltage_domain=vd)
diff --git a/configs/example/hmctest.py b/configs/example/hmctest.py
index f4c0a2c..429ea4a 100644
--- a/configs/example/hmctest.py
+++ b/configs/example/hmctest.py
@@ -50,7 +50,7 @@
system = System()
# use timing mode for the interaction between requestor-responder ports
system.mem_mode = 'timing'
- # set the clock fequency of the system
+ # set the clock frequency of the system
clk = '100GHz'
vd = VoltageDomain(voltage='1V')
system.clk_domain = SrcClockDomain(clock=clk, voltage_domain=vd)
diff --git a/configs/example/lupv/run_lupv.py b/configs/example/lupv/run_lupv.py
index 93eaa86..e87d392 100644
--- a/configs/example/lupv/run_lupv.py
+++ b/configs/example/lupv/run_lupv.py
@@ -36,7 +36,6 @@
import m5
from m5.objects import Root
-from gem5.runtime import get_runtime_isa
from gem5.components.boards.experimental.lupv_board import LupvBoard
from gem5.components.memory.single_channel import SingleChannelDDR3_1600
from gem5.components.processors.simple_processor import SimpleProcessor
@@ -83,11 +82,11 @@
# Setup a single core Processor.
if args.cpu_type == "atomic":
processor = SimpleProcessor(
- cpu_type=CPUTypes.ATOMIC, num_cores=args.num_cpus
+ cpu_type=CPUTypes.ATOMIC, num_cores=args.num_cpus, isa=ISA.RISCV
)
elif args.cpu_type == "timing":
processor = SimpleProcessor(
- cpu_type=CPUTypes.TIMING, num_cores=args.num_cpus
+ cpu_type=CPUTypes.TIMING, num_cores=args.num_cpus, isa=ISA.RISCV
)
# Setup the board.
@@ -106,7 +105,7 @@
# Begin running of the simulation.
-print("Running with ISA: " + get_runtime_isa().name)
+print("Running with ISA: " + processor.get_isa().name)
print()
root = Root(full_system=True, system=board)
m5.instantiate()
diff --git a/configs/example/noc_config/2x4.py b/configs/example/noc_config/2x4.py
index fe2522b..2d10da6 100644
--- a/configs/example/noc_config/2x4.py
+++ b/configs/example/noc_config/2x4.py
@@ -60,6 +60,10 @@
class NoC_Params(CHI_config.CHI_HNF.NoC_Params):
router_list = [1, 2, 5, 6]
+class CHI_MN(CHI_config.CHI_MN):
+ class NoC_Params(CHI_config.CHI_MN.NoC_Params):
+ router_list = [4]
+
class CHI_SNF_MainMem(CHI_config.CHI_SNF_MainMem):
class NoC_Params(CHI_config.CHI_SNF_MainMem.NoC_Params):
router_list = [0, 4]
diff --git a/configs/example/ruby_gpu_random_test.py b/configs/example/ruby_gpu_random_test.py
index a5d9a49..029a97d 100644
--- a/configs/example/ruby_gpu_random_test.py
+++ b/configs/example/ruby_gpu_random_test.py
@@ -79,7 +79,7 @@
help="Random seed number. Default value (i.e., 0) means \
using runtime-specific value")
parser.add_argument("--log-file", type=str, default="gpu-ruby-test.log")
-parser.add_argument("--num-dmas", type=int, default=0,
+parser.add_argument("--num-dmas", type=int, default=None,
help="The number of DMA engines to use in tester config.")
args = parser.parse_args()
@@ -108,7 +108,7 @@
args.wf_size = 1
args.wavefronts_per_cu = 1
args.num_cpus = 1
- args.num_dmas = 1
+ n_DMAs = 1
args.cu_per_sqc = 1
args.cu_per_scalar_cache = 1
args.num_compute_units = 1
@@ -117,7 +117,7 @@
args.wf_size = 16
args.wavefronts_per_cu = 4
args.num_cpus = 4
- args.num_dmas = 2
+ n_DMAs = 2
args.cu_per_sqc = 4
args.cu_per_scalar_cache = 4
args.num_compute_units = 4
@@ -126,11 +126,19 @@
args.wf_size = 32
args.wavefronts_per_cu = 4
args.num_cpus = 4
- args.num_dmas = 4
+ n_DMAs = 4
args.cu_per_sqc = 4
args.cu_per_scalar_cache = 4
args.num_compute_units = 8
+# Number of DMA engines
+if not(args.num_dmas is None):
+ n_DMAs = args.num_dmas
+ # currently the tester does not support requests returned as
+ # aliased, thus we need num_dmas to be 0 for it
+ if not(args.num_dmas == 0):
+ print("WARNING: num_dmas != 0 not supported with VIPER")
+
#
# Set address range - 2 options
# level 0: small
@@ -173,9 +181,6 @@
# For now we're testing only GPU protocol, so we force num_cpus to be 0
args.num_cpus = 0
-# Number of DMA engines
-n_DMAs = args.num_dmas
-
# Number of CUs
n_CUs = args.num_compute_units
@@ -234,11 +239,12 @@
#
# Make generic DMA sequencer for Ruby to use
#
-dma_devices = [TesterDma()] * n_DMAs
-system.piobus = IOXBar()
-for _, dma_device in enumerate(dma_devices):
- dma_device.pio = system.piobus.mem_side_ports
-system.dma_devices = dma_devices
+if n_DMAs > 0:
+ dma_devices = [TesterDma()] * n_DMAs
+ system.piobus = IOXBar()
+ for _, dma_device in enumerate(dma_devices):
+ dma_device.pio = system.piobus.mem_side_ports
+ system.dma_devices = dma_devices
#
# Create the Ruby system
@@ -250,7 +256,8 @@
# size of system.cpu
cpu_list = [ system.cpu ] * args.num_cpus
Ruby.create_system(args, full_system = False,
- system = system, dma_ports = system.dma_devices,
+ system = system,
+ dma_ports = system.dma_devices if n_DMAs > 0 else [],
cpus = cpu_list)
#
@@ -275,7 +282,10 @@
for i, ruby_port in enumerate(system.ruby._cpu_ports):
ruby_port.no_retry_on_stall = True
ruby_port.using_ruby_tester = True
- ruby_port.mem_request_port = system.piobus.cpu_side_ports
+
+ # piobus is only created if there are DMAs
+ if n_DMAs > 0:
+ ruby_port.mem_request_port = system.piobus.cpu_side_ports
if i < n_CUs:
tester.cu_vector_ports = ruby_port.in_ports
diff --git a/configs/example/se.py b/configs/example/se.py
index a3b5cb9..3a8203d 100644
--- a/configs/example/se.py
+++ b/configs/example/se.py
@@ -200,6 +200,7 @@
if ObjectList.is_kvm_cpu(CPUClass) or ObjectList.is_kvm_cpu(FutureClass):
if buildEnv['TARGET_ISA'] == 'x86':
system.kvm_vm = KvmVM()
+ system.m5ops_base = 0xffff0000
for process in multiprocesses:
process.useArchPT = True
process.kvmInSE = True
diff --git a/configs/learning_gem5/part1/simple.py b/configs/learning_gem5/part1/simple.py
index 235165b..7810c3d 100644
--- a/configs/learning_gem5/part1/simple.py
+++ b/configs/learning_gem5/part1/simple.py
@@ -43,7 +43,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/learning_gem5/part1/two_level.py b/configs/learning_gem5/part1/two_level.py
index 591be0c..7a7956c 100644
--- a/configs/learning_gem5/part1/two_level.py
+++ b/configs/learning_gem5/part1/two_level.py
@@ -70,7 +70,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/learning_gem5/part2/simple_cache.py b/configs/learning_gem5/part2/simple_cache.py
index adda386..dc3a162 100644
--- a/configs/learning_gem5/part2/simple_cache.py
+++ b/configs/learning_gem5/part2/simple_cache.py
@@ -39,7 +39,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/learning_gem5/part2/simple_memobj.py b/configs/learning_gem5/part2/simple_memobj.py
index 49b0846..eaceae9 100644
--- a/configs/learning_gem5/part2/simple_memobj.py
+++ b/configs/learning_gem5/part2/simple_memobj.py
@@ -39,7 +39,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/learning_gem5/part3/ruby_test.py b/configs/learning_gem5/part3/ruby_test.py
index d0c3910..05096ec 100644
--- a/configs/learning_gem5/part3/ruby_test.py
+++ b/configs/learning_gem5/part3/ruby_test.py
@@ -44,7 +44,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/learning_gem5/part3/simple_ruby.py b/configs/learning_gem5/part3/simple_ruby.py
index 2e65ebd..3d9fe9a 100644
--- a/configs/learning_gem5/part3/simple_ruby.py
+++ b/configs/learning_gem5/part3/simple_ruby.py
@@ -53,7 +53,7 @@
# create the system we are going to simulate
system = System()
-# Set the clock fequency of the system (and all of its children)
+# Set the clock frequency of the system (and all of its children)
system.clk_domain = SrcClockDomain()
system.clk_domain.clock = '1GHz'
system.clk_domain.voltage_domain = VoltageDomain()
diff --git a/configs/network/Network.py b/configs/network/Network.py
index 91f0076..e5a86f6 100644
--- a/configs/network/Network.py
+++ b/configs/network/Network.py
@@ -80,6 +80,10 @@
"--garnet-deadlock-threshold", action="store",
type=int, default=50000,
help="network-level deadlock threshold.")
+ parser.add_argument("--simple-physical-channels", action="store_true",
+ default=False,
+ help="""SimpleNetwork links uses a separate physical
+ channel for each virtual network""")
def create_network(options, ruby):
@@ -187,6 +191,9 @@
extLink.int_cred_bridge = int_cred_bridges
if options.network == "simple":
+ if options.simple_physical_channels:
+ network.physical_vnets_channels = \
+ [1] * int(network.number_of_virtual_networks)
network.setup_buffers()
if InterfaceClass != None:
diff --git a/configs/nvm/sweep.py b/configs/nvm/sweep.py
index 7693fb8..09371f9 100644
--- a/configs/nvm/sweep.py
+++ b/configs/nvm/sweep.py
@@ -106,14 +106,14 @@
# controller with an NVM interface, check to be sure
if not isinstance(system.mem_ctrls[0], m5.objects.MemCtrl):
fatal("This script assumes the controller is a MemCtrl subclass")
-if not isinstance(system.mem_ctrls[0].nvm, m5.objects.NVMInterface):
+if not isinstance(system.mem_ctrls[0].dram, m5.objects.NVMInterface):
fatal("This script assumes the memory is a NVMInterface class")
# there is no point slowing things down by saving any data
-system.mem_ctrls[0].nvm.null = True
+system.mem_ctrls[0].dram.null = True
# Set the address mapping based on input argument
-system.mem_ctrls[0].nvm.addr_mapping = args.addr_map
+system.mem_ctrls[0].dram.addr_mapping = args.addr_map
# stay in each state for 0.25 ms, long enough to warm things up, and
# short enough to avoid hitting a refresh
@@ -124,21 +124,21 @@
# the DRAM maximum bandwidth to ensure that it is saturated
# get the number of regions
-nbr_banks = system.mem_ctrls[0].nvm.banks_per_rank.value
+nbr_banks = system.mem_ctrls[0].dram.banks_per_rank.value
# determine the burst length in bytes
-burst_size = int((system.mem_ctrls[0].nvm.devices_per_rank.value *
- system.mem_ctrls[0].nvm.device_bus_width.value *
- system.mem_ctrls[0].nvm.burst_length.value) / 8)
+burst_size = int((system.mem_ctrls[0].dram.devices_per_rank.value *
+ system.mem_ctrls[0].dram.device_bus_width.value *
+ system.mem_ctrls[0].dram.burst_length.value) / 8)
# next, get the page size in bytes
-buffer_size = system.mem_ctrls[0].nvm.devices_per_rank.value * \
- system.mem_ctrls[0].nvm.device_rowbuffer_size.value
+buffer_size = system.mem_ctrls[0].dram.devices_per_rank.value * \
+ system.mem_ctrls[0].dram.device_rowbuffer_size.value
# match the maximum bandwidth of the memory, the parameter is in seconds
# and we need it in ticks (ps)
-itt = system.mem_ctrls[0].nvm.tBURST.value * 1000000000000
+itt = system.mem_ctrls[0].dram.tBURST.value * 1000000000000
# assume we start at 0
max_addr = mem_range.end
@@ -179,7 +179,7 @@
0, max_addr, burst_size, int(itt), int(itt),
args.rd_perc, 0,
num_seq_pkts, buffer_size, nbr_banks, bank,
- addr_map, args.nvm_ranks)
+ addr_map, args.dram_ranks)
yield system.tgen.createExit(0)
system.tgen.start(trace())
diff --git a/configs/nvm/sweep_hybrid.py b/configs/nvm/sweep_hybrid.py
index 4594f78..6bccdef 100644
--- a/configs/nvm/sweep_hybrid.py
+++ b/configs/nvm/sweep_hybrid.py
@@ -117,8 +117,8 @@
# the following assumes that we are using the native controller
# with NVM and DRAM interfaces, check to be sure
-if not isinstance(system.mem_ctrls[0], m5.objects.MemCtrl):
- fatal("This script assumes the controller is a MemCtrl subclass")
+if not isinstance(system.mem_ctrls[0], m5.objects.HeteroMemCtrl):
+ fatal("This script assumes the controller is a HeteroMemCtrl subclass")
if not isinstance(system.mem_ctrls[0].dram, m5.objects.DRAMInterface):
fatal("This script assumes the first memory is a DRAMInterface subclass")
if not isinstance(system.mem_ctrls[0].nvm, m5.objects.NVMInterface):
diff --git a/configs/ruby/CHI.py b/configs/ruby/CHI.py
index e4a2477..c94dc94 100644
--- a/configs/ruby/CHI.py
+++ b/configs/ruby/CHI.py
@@ -43,6 +43,7 @@
default=None,
help="NoC config. parameters and bindings. "
"Required for CustomMesh topology")
+ parser.add_argument("--enable-dvm", default=False, action="store_true")
def read_config_file(file):
''' Read file as a module and return it '''
@@ -65,6 +66,13 @@
if options.num_l3caches < 1:
m5.fatal('--num-l3caches must be at least 1')
+ if full_system and options.enable_dvm:
+ if len(cpus) <= 1:
+ m5.fatal("--enable-dvm can't be used with a single CPU")
+ for cpu in cpus:
+ for decoder in cpu.decoder:
+ decoder.dvm_enabled = True
+
# read specialized classes from config file if provided
if options.chi_config:
chi_defs = read_config_file(options.chi_config)
@@ -79,6 +87,7 @@
# Node types
CHI_RNF = chi_defs.CHI_RNF
CHI_HNF = chi_defs.CHI_HNF
+ CHI_MN = chi_defs.CHI_MN
CHI_SNF_MainMem = chi_defs.CHI_SNF_MainMem
CHI_SNF_BootMem = chi_defs.CHI_SNF_BootMem
CHI_RNI_DMA = chi_defs.CHI_RNI_DMA
@@ -140,6 +149,14 @@
network_nodes.append(rnf)
network_cntrls.extend(rnf.getNetworkSideControllers())
+ # Creates one Misc Node
+ ruby_system.mn = [ CHI_MN(ruby_system, [cpu.l1d for cpu in cpus]) ]
+ for mn in ruby_system.mn:
+ all_cntrls.extend(mn.getAllControllers())
+ network_nodes.append(mn)
+ network_cntrls.extend(mn.getNetworkSideControllers())
+ assert(mn.getAllControllers() == mn.getNetworkSideControllers())
+
# Look for other memories
other_memories = []
if bootmem:
@@ -156,8 +173,9 @@
for m in other_memories:
sysranges.append(m.range)
+ hnf_list = [i for i in range(options.num_l3caches)]
CHI_HNF.createAddrRanges(sysranges, system.cache_line_size.value,
- options.num_l3caches)
+ hnf_list)
ruby_system.hnf = [ CHI_HNF(i, ruby_system, HNFCache, None)
for i in range(options.num_l3caches) ]
diff --git a/configs/ruby/CHI_config.py b/configs/ruby/CHI_config.py
index 097f367..a4b01ca 100644
--- a/configs/ruby/CHI_config.py
+++ b/configs/ruby/CHI_config.py
@@ -230,6 +230,9 @@
self.number_of_TBEs = 16
self.number_of_repl_TBEs = 16
self.number_of_snoop_TBEs = 4
+ self.number_of_DVM_TBEs = 16
+ self.number_of_DVM_snoop_TBEs = 4
+
self.unify_repl_TBEs = False
class CHI_L2Controller(CHI_Cache_Controller):
@@ -262,6 +265,8 @@
self.number_of_TBEs = 32
self.number_of_repl_TBEs = 32
self.number_of_snoop_TBEs = 16
+ self.number_of_DVM_TBEs = 1 # should not receive any dvm
+ self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
class CHI_HNFController(CHI_Cache_Controller):
@@ -295,8 +300,41 @@
self.number_of_TBEs = 32
self.number_of_repl_TBEs = 32
self.number_of_snoop_TBEs = 1 # should not receive any snoop
+ self.number_of_DVM_TBEs = 1 # should not receive any dvm
+ self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
+class CHI_MNController(MiscNode_Controller):
+ '''
+ Default parameters for a Misc Node
+ '''
+
+ def __init__(self, ruby_system, addr_range, l1d_caches,
+ early_nonsync_comp):
+ super(CHI_MNController, self).__init__(
+ version = Versions.getVersion(MiscNode_Controller),
+ ruby_system = ruby_system,
+ mandatoryQueue = MessageBuffer(),
+ triggerQueue = TriggerMessageBuffer(),
+ retryTriggerQueue = TriggerMessageBuffer(),
+ schedRspTriggerQueue = TriggerMessageBuffer(),
+ reqRdy = TriggerMessageBuffer(),
+ snpRdy = TriggerMessageBuffer(),
+ )
+ # Set somewhat large number since we really a lot on internal
+ # triggers. To limit the controller performance, tweak other
+ # params such as: input port buffer size, cache banks, and output
+ # port latency
+ self.transitions_per_cycle = 1024
+ self.addr_ranges = [addr_range]
+ # 16 total transaction buffer entries, but 1 is reserved for DVMNonSync
+ self.number_of_DVM_TBEs = 16
+ self.number_of_non_sync_TBEs = 1
+ self.early_nonsync_comp = early_nonsync_comp
+
+ # "upstream_destinations" = targets for DVM snoops
+ self.upstream_destinations = l1d_caches
+
class CHI_DMAController(CHI_Cache_Controller):
'''
Default parameters for a DMA controller
@@ -333,6 +371,8 @@
self.number_of_TBEs = 16
self.number_of_repl_TBEs = 1
self.number_of_snoop_TBEs = 1 # should not receive any snoop
+ self.number_of_DVM_TBEs = 1 # should not receive any dvm
+ self.number_of_DVM_snoop_TBEs = 1 # should not receive any dvm
self.unify_repl_TBEs = False
class CPUSequencerWrapper:
@@ -486,15 +526,14 @@
'''HNFs may also define the 'pairing' parameter to allow pairing'''
pairing = None
- _addr_ranges = []
+ _addr_ranges = {}
@classmethod
- def createAddrRanges(cls, sys_mem_ranges, cache_line_size, num_hnfs):
+ def createAddrRanges(cls, sys_mem_ranges, cache_line_size, hnfs):
# Create the HNFs interleaved addr ranges
block_size_bits = int(math.log(cache_line_size, 2))
- cls._addr_ranges = []
- llc_bits = int(math.log(num_hnfs, 2))
+ llc_bits = int(math.log(len(hnfs), 2))
numa_bit = block_size_bits + llc_bits - 1
- for i in range(num_hnfs):
+ for i, hnf in enumerate(hnfs):
ranges = []
for r in sys_mem_ranges:
addr_range = AddrRange(r.start, size = r.size(),
@@ -502,7 +541,7 @@
intlvBits = llc_bits,
intlvMatch = i)
ranges.append(addr_range)
- cls._addr_ranges.append((ranges, numa_bit, i))
+ cls._addr_ranges[hnf] = (ranges, numa_bit)
@classmethod
def getAddrRanges(cls, hnf_idx):
@@ -514,10 +553,9 @@
def __init__(self, hnf_idx, ruby_system, llcache_type, parent):
super(CHI_HNF, self).__init__(ruby_system)
- addr_ranges,intlvHighBit,intlvMatch = self.getAddrRanges(hnf_idx)
+ addr_ranges,intlvHighBit = self.getAddrRanges(hnf_idx)
# All ranges should have the same interleaving
assert(len(addr_ranges) >= 1)
- assert(intlvMatch == hnf_idx)
ll_cache = llcache_type(start_index_bit = intlvHighBit + 1)
self._cntrl = CHI_HNFController(ruby_system, ll_cache, NULL,
@@ -537,6 +575,40 @@
return [self._cntrl]
+class CHI_MN(CHI_Node):
+ '''
+ Encapsulates a Misc Node controller.
+ '''
+
+ class NoC_Params(CHI_Node.NoC_Params):
+ '''HNFs may also define the 'pairing' parameter to allow pairing'''
+ pairing = None
+
+
+ # The CHI controller can be a child of this object or another if
+ # 'parent' if specified
+ def __init__(self, ruby_system, l1d_caches, early_nonsync_comp=False):
+ super(CHI_MN, self).__init__(ruby_system)
+
+ # MiscNode has internal address range starting at 0
+ addr_range = AddrRange(0, size = "1kB")
+
+ self._cntrl = CHI_MNController(ruby_system, addr_range, l1d_caches,
+ early_nonsync_comp)
+
+ self.cntrl = self._cntrl
+
+ self.connectController(self._cntrl)
+
+ def connectController(self, cntrl):
+ CHI_Node.connectController(self, cntrl)
+
+ def getAllControllers(self):
+ return [self._cntrl]
+
+ def getNetworkSideControllers(self):
+ return [self._cntrl]
+
class CHI_SNF_Base(CHI_Node):
'''
Creates CHI node controllers for the memory controllers
diff --git a/configs/ruby/GPU_VIPER.py b/configs/ruby/GPU_VIPER.py
index f8a7386..dc99429 100644
--- a/configs/ruby/GPU_VIPER.py
+++ b/configs/ruby/GPU_VIPER.py
@@ -34,6 +34,8 @@
from m5.util import addToPath
from .Ruby import create_topology
from .Ruby import send_evicts
+from common import ObjectList
+from common import MemConfig
from common import FileSystemConfig
addToPath('../')
@@ -443,6 +445,66 @@
dir_cntrl.triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.L3triggerQueue = MessageBuffer(ordered = True)
+ dir_cntrl.requestToMemory = MessageBuffer(ordered = True)
+ dir_cntrl.responseFromMemory = MessageBuffer(ordered = True)
+
+ dir_cntrl.requestFromDMA = MessageBuffer(ordered=True)
+ dir_cntrl.requestFromDMA.in_port = network.out_port
+
+ dir_cntrl.responseToDMA = MessageBuffer()
+ dir_cntrl.responseToDMA.out_port = network.in_port
+
+ exec("ruby_system.dir_cntrl%d = dir_cntrl" % i)
+ dir_cntrl_nodes.append(dir_cntrl)
+
+ return dir_cntrl_nodes
+
+def construct_gpudirs(options, system, ruby_system, network):
+
+ dir_cntrl_nodes = []
+ mem_ctrls = []
+
+ xor_low_bit = 0
+
+ # For an odd number of CPUs, still create the right number of controllers
+ TCC_bits = int(math.log(options.num_tccs, 2))
+
+ dir_bits = int(math.log(options.dgpu_num_dirs, 2))
+ block_size_bits = int(math.log(options.cacheline_size, 2))
+ numa_bit = block_size_bits + dir_bits - 1
+
+ gpu_mem_range = AddrRange(0, size = options.dgpu_mem_size)
+ for i in range(options.dgpu_num_dirs):
+ addr_range = m5.objects.AddrRange(gpu_mem_range.start,
+ size = gpu_mem_range.size(),
+ intlvHighBit = numa_bit,
+ intlvBits = dir_bits,
+ intlvMatch = i,
+ xorHighBit = xor_low_bit)
+
+ dir_cntrl = DirCntrl(noTCCdir = True, TCC_select_num_bits = TCC_bits)
+ dir_cntrl.create(options, [addr_range], ruby_system, system)
+ dir_cntrl.number_of_TBEs = options.num_tbes
+ dir_cntrl.useL3OnWT = False
+
+ # Connect the Directory controller to the ruby network
+ dir_cntrl.requestFromCores = MessageBuffer(ordered = True)
+ dir_cntrl.requestFromCores.in_port = network.out_port
+
+ dir_cntrl.responseFromCores = MessageBuffer()
+ dir_cntrl.responseFromCores.in_port = network.out_port
+
+ dir_cntrl.unblockFromCores = MessageBuffer()
+ dir_cntrl.unblockFromCores.in_port = network.out_port
+
+ dir_cntrl.probeToCore = MessageBuffer()
+ dir_cntrl.probeToCore.out_port = network.in_port
+
+ dir_cntrl.responseToCore = MessageBuffer()
+ dir_cntrl.responseToCore.out_port = network.in_port
+
+ dir_cntrl.triggerQueue = MessageBuffer(ordered = True)
+ dir_cntrl.L3triggerQueue = MessageBuffer(ordered = True)
dir_cntrl.requestToMemory = MessageBuffer()
dir_cntrl.responseFromMemory = MessageBuffer()
@@ -455,10 +517,28 @@
dir_cntrl.requestToMemory = MessageBuffer()
dir_cntrl.responseFromMemory = MessageBuffer()
- exec("ruby_system.dir_cntrl%d = dir_cntrl" % i)
- dir_cntrl_nodes.append(dir_cntrl)
+ # Create memory controllers too
+ mem_type = ObjectList.mem_list.get(options.dgpu_mem_type)
+ dram_intf = MemConfig.create_mem_intf(mem_type, gpu_mem_range, i,
+ int(math.log(options.dgpu_num_dirs, 2)), options.cacheline_size,
+ xor_low_bit)
+ if issubclass(mem_type, DRAMInterface):
+ mem_ctrl = m5.objects.MemCtrl(dram = dram_intf)
+ else:
+ mem_ctrl = dram_intf
- return dir_cntrl_nodes
+ mem_ctrl.port = dir_cntrl.memory_out_port
+ mem_ctrl.dram.enable_dram_powerdown = False
+ dir_cntrl.addr_ranges = dram_intf.range
+
+ # Append
+ exec("system.ruby.gpu_dir_cntrl%d = dir_cntrl" % i)
+ dir_cntrl_nodes.append(dir_cntrl)
+ mem_ctrls.append(mem_ctrl)
+
+ system.gpu_mem_ctrls = mem_ctrls
+
+ return dir_cntrl_nodes, mem_ctrls
def construct_corepairs(options, system, ruby_system, network):
diff --git a/configs/ruby/Ruby.py b/configs/ruby/Ruby.py
index 631c65c..ba94c15 100644
--- a/configs/ruby/Ruby.py
+++ b/configs/ruby/Ruby.py
@@ -130,7 +130,7 @@
if len(system.mem_ranges) > 1:
crossbar = IOXBar()
crossbars.append(crossbar)
- dir_cntrl.memory = crossbar.cpu_side_ports
+ dir_cntrl.memory_out_port = crossbar.cpu_side_ports
dir_ranges = []
for r in system.mem_ranges:
@@ -152,7 +152,7 @@
if crossbar != None:
mem_ctrl.port = crossbar.mem_side_ports
else:
- mem_ctrl.port = dir_cntrl.memory
+ mem_ctrl.port = dir_cntrl.memory_out_port
# Enable low-power DRAM states if option is set
if issubclass(mem_type, DRAMInterface):
diff --git a/configs/topologies/CustomMesh.py b/configs/topologies/CustomMesh.py
index 70bf55d..2519bdd 100644
--- a/configs/topologies/CustomMesh.py
+++ b/configs/topologies/CustomMesh.py
@@ -239,6 +239,7 @@
# classify nodes into different types
rnf_nodes = []
hnf_nodes = []
+ mn_nodes = []
mem_nodes = []
io_mem_nodes = []
rni_dma_nodes = []
@@ -248,6 +249,7 @@
# the same base type.
rnf_params = None
hnf_params = None
+ mn_params = None
mem_params = None
io_mem_params = None
rni_dma_params = None
@@ -264,6 +266,9 @@
elif isinstance(n, CHI.CHI_HNF):
hnf_nodes.append(n)
hnf_params = check_same(type(n).NoC_Params, hnf_params)
+ elif isinstance(n, CHI.CHI_MN):
+ mn_nodes.append(n)
+ mn_params = check_same(type(n).NoC_Params, mn_params)
elif isinstance(n, CHI.CHI_SNF_MainMem):
mem_nodes.append(n)
mem_params = check_same(type(n).NoC_Params, mem_params)
@@ -298,6 +303,9 @@
# Place CHI_HNF on the mesh
self.distributeNodes(hnf_params, hnf_nodes)
+ # Place CHI_MN on the mesh
+ self.distributeNodes(options, mn_params, mn_nodes)
+
# Place CHI_SNF_MainMem on the mesh
self.distributeNodes(mem_params, mem_nodes)
diff --git a/ext/Kconfiglib/.gitignore b/ext/Kconfiglib/.gitignore
new file mode 100644
index 0000000..4a18d79
--- /dev/null
+++ b/ext/Kconfiglib/.gitignore
@@ -0,0 +1,4 @@
+*.py[co]
+build/
+*.egg-info/
+dist/
diff --git a/ext/Kconfiglib/LICENSE.txt b/ext/Kconfiglib/LICENSE.txt
new file mode 100644
index 0000000..8b31efc
--- /dev/null
+++ b/ext/Kconfiglib/LICENSE.txt
@@ -0,0 +1,5 @@
+Copyright (c) 2011-2019, Ulf Magnusson <ulfalizer@gmail.com>
+
+Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
diff --git a/ext/Kconfiglib/MANIFEST.in b/ext/Kconfiglib/MANIFEST.in
new file mode 100644
index 0000000..881a794
--- /dev/null
+++ b/ext/Kconfiglib/MANIFEST.in
@@ -0,0 +1,2 @@
+# Include the license file in source distributions
+include LICENSE.txt
diff --git a/ext/Kconfiglib/README.rst b/ext/Kconfiglib/README.rst
new file mode 100644
index 0000000..b59f04e
--- /dev/null
+++ b/ext/Kconfiglib/README.rst
@@ -0,0 +1,841 @@
+.. contents:: Table of contents
+ :backlinks: none
+
+News
+----
+
+Dependency loop with recent linux-next kernels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To fix issues with dependency loops on recent linux-next kernels, apply `this
+patch <https://www.spinics.net/lists/linux-kbuild/msg23455.html>`_. Hopefully,
+it will be in ``linux-next`` soon.
+
+``windows-curses`` is no longer automatically installed on Windows
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Starting with Kconfiglib 13.0.0, the `windows-curses
+<https://github.com/zephyrproject-rtos/windows-curses>`__ package is no longer
+automatically installed on Windows, and needs to be installed manually for the
+terminal ``menuconfig`` to work.
+
+This fixes installation of Kconfiglib on MSYS2, which is not compatible with
+``windows-curses``. See `this issue
+<https://github.com/ulfalizer/Kconfiglib/issues/77>`__.
+
+The ``menuconfig`` now shows a hint re. installing ``windows-curses`` when the
+``curses`` module can't be imported on Windows.
+
+Sorry if this change caused problems!
+
+Overview
+--------
+
+Kconfiglib is a `Kconfig
+<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
+implementation in Python 2/3. It started out as a helper library, but now has a
+enough functionality to also work well as a standalone Kconfig implementation
+(including `terminal and GUI menuconfig interfaces <Menuconfig interfaces_>`_
+and `Kconfig extensions`_).
+
+The entire library is contained in `kconfiglib.py
+<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_. The
+bundled scripts are implemented on top of it. Implementing your own scripts
+should be relatively easy, if needed.
+
+Kconfiglib is used exclusively by e.g. the `Zephyr
+<https://www.zephyrproject.org/>`__, `esp-idf
+<https://github.com/espressif/esp-idf>`__, and `ACRN
+<https://projectacrn.org/>`__ projects. It is also used for many small helper
+scripts in various projects.
+
+Since Kconfiglib is based around a library, it can be used e.g. to generate a
+`Kconfig cross-reference
+<https://docs.zephyrproject.org/latest/reference/kconfig/index.html>`_, using
+the same robust Kconfig parser used for other Kconfig tools, instead of brittle
+ad-hoc parsing. The documentation generation script can be found `here
+<https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`__.
+
+Kconfiglib implements the recently added `Kconfig preprocessor
+<https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__.
+For backwards compatibility, environment variables can be referenced both as
+``$(FOO)`` (the new syntax) and as ``$FOO`` (the old syntax). The old syntax is
+deprecated, but will probably be supported for a long time, as it's needed to
+stay compatible with older Linux kernels. The major version will be increased
+if support is ever dropped. Using the old syntax with an undefined environment
+variable keeps the string as is.
+
+Note: See `this issue <https://github.com/ulfalizer/Kconfiglib/issues/47>`__ if
+you run into a "macro expanded to blank string" error with kernel 4.18+.
+
+See `this page
+<https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
+Kconfig tips and best practices.
+
+Installation
+------------
+
+Installation with pip
+~~~~~~~~~~~~~~~~~~~~~
+
+Kconfiglib is available on `PyPI <https://pypi.python.org/pypi/kconfiglib/>`_ and can be
+installed with e.g.
+
+.. code::
+
+ $ pip(3) install kconfiglib
+
+Microsoft Windows is supported.
+
+The ``pip`` installation will give you both the base library and the following
+executables. All but two (``genconfig`` and ``setconfig``) mirror functionality
+available in the C tools.
+
+- `menuconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
+
+- `guiconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_
+
+- `oldconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/oldconfig.py>`_
+
+- `olddefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/olddefconfig.py>`_
+
+- `savedefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/savedefconfig.py>`_
+
+- `defconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/defconfig.py>`_
+
+- `alldefconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/alldefconfig.py>`_
+
+- `allnoconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_
+
+- `allmodconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allmodconfig.py>`_
+
+- `allyesconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
+
+- `listnewconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/listnewconfig.py>`_
+
+- `genconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/genconfig.py>`_
+
+- `setconfig <https://github.com/ulfalizer/Kconfiglib/blob/master/setconfig.py>`_
+
+``genconfig`` is intended to be run at build time. It generates a C header from
+the configuration and (optionally) information that can be used to rebuild only
+files that reference Kconfig symbols that have changed value.
+
+Starting with Kconfiglib version 12.2.0, all utilities are compatible with both
+Python 2 and Python 3. Previously, ``menuconfig.py`` only ran under Python 3
+(i.e., it's now more backwards compatible than before).
+
+**Note:** If you install Kconfiglib with ``pip``'s ``--user`` flag, make sure
+that your ``PATH`` includes the directory where the executables end up. You can
+list the installed files with ``pip(3) show -f kconfiglib``.
+
+All releases have a corresponding tag in the git repository, e.g. ``v14.1.0``
+(the latest version).
+
+`Semantic versioning <http://semver.org/>`_ is used. There's been ten small
+changes to the behavior of the API, a Windows packaging change, and a hashbang
+change to use ``python3``
+(`1 <https://github.com/ulfalizer/Kconfiglib/commit/e8b4ecb6ff6ccc1c7be0818314fbccda2ef2b2ee>`_,
+`2 <https://github.com/ulfalizer/Kconfiglib/commit/db633015a4d7b0ba1e882f665e191f350932b2af>`_,
+`3 <https://github.com/ulfalizer/Kconfiglib/commit/8983f7eb297dd614faf0beee3129559bc8ba338e>`_,
+`4 <https://github.com/ulfalizer/Kconfiglib/commit/cbf32e29a130d22bc734b7778e6304ac9df2a3e8>`_,
+`5 <https://github.com/ulfalizer/Kconfiglib/commit/eb6c21a9b33a2d6e2bed9882d4f930d0cab2f03b>`_,
+`6 <https://github.com/ulfalizer/Kconfiglib/commit/c19fc11355b13d75d97286402c7a933fb23d3b70>`_,
+`7 <https://github.com/ulfalizer/Kconfiglib/commit/7a428aa415606820a44291f475248b08e3952c4b>`_,
+`8 <https://github.com/ulfalizer/Kconfiglib/commit/f247ddf618ad29718e5efd3e69f8baf75d4d347b>`_,
+`9 <https://github.com/ulfalizer/Kconfiglib/commit/4fed39d9271ceb68be4157ab3f96a45b94f77dc0>`_,
+`10 <https://github.com/ulfalizer/Kconfiglib/commit/55bc8c380869ea663092212e8fe388ad7abae596>`_,
+`Windows packaging change <https://github.com/ulfalizer/Kconfiglib/commit/21b4c1e3b6e2867b9a0788d21a358f6b1f581d86>`_,
+`Python 3 hashbang change <https://github.com/ulfalizer/Kconfiglib/commit/9e0a8d29fa76adcb3f27bb2e20f16fefc2a8591e>`_),
+which is why the major version is at 14 rather than 2. I do major version bumps
+for all behavior changes, even tiny ones, and most of these were fixes for baby
+issues in the early days of the Kconfiglib 2 API.
+
+Manual installation
+~~~~~~~~~~~~~~~~~~~
+
+Just drop ``kconfiglib.py`` and the scripts you want somewhere. There are no
+third-party dependencies, but the terminal ``menuconfig`` won't work on Windows
+unless a package like `windows-curses
+<https://github.com/zephyrproject-rtos/windows-curses>`__ is installed.
+
+Installation for the Linux kernel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See the module docstring at the top of `kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
+
+Python version compatibility (2.7/3.2+)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Kconfiglib and all utilities run under both Python 2.7 and Python 3.2 and
+later. The code mostly uses basic Python features and has no third-party
+dependencies, so keeping it backwards-compatible is pretty low effort.
+
+The 3.2 requirement comes from ``argparse``. ``format()`` with unnumbered
+``{}`` is used as well.
+
+A recent Python 3 version is recommended if you have a choice, as it'll give
+you better Unicode handling.
+
+Getting started
+---------------
+
+1. `Install <Installation_>`_ the library and the utilities.
+
+2. Write `Kconfig
+ <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-language.rst>`__
+ files that describe the available configuration options. See `this page
+ <https://docs.zephyrproject.org/latest/guides/kconfig/tips.html>`__ for some
+ general Kconfig advice.
+
+3. Generate an initial configuration with e.g. ``menuconfig``/``guiconfig`` or
+ ``alldefconfig``. The configuration is saved as ``.config`` by default.
+
+ For more advanced projects, the ``defconfig`` utility can be used to
+ generate the initial configuration from an existing configuration file.
+ Usually, this existing configuration file would be a minimal configuration
+ file, as generated by e.g. ``savedefconfig``.
+
+4. Run ``genconfig`` to generate a header file. By default, it is saved as
+ ``config.h``.
+
+ Normally, ``genconfig`` would be run automatically as part of the build.
+
+ Before writing a header file or other configuration output, Kconfiglib
+ compares the old contents of the file against the new contents. If there's
+ no change, the write is skipped. This avoids updating file metadata like the
+ modification time, and might save work depending on your build setup.
+
+ Adding new configuration output formats should be relatively straightforward.
+ See the implementation of ``write_config()`` in `kconfiglib.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
+ The documentation for the ``Symbol.config_string`` property has some tips as
+ well.
+
+5. To update an old ``.config`` file after the Kconfig files have changed (e.g.
+ to add new options), run ``oldconfig`` (prompts for values for new options)
+ or ``olddefconfig`` (gives new options their default value). Entering the
+ ``menuconfig`` or ``guiconfig`` interface and saving the configuration will
+ also update it (the configuration interfaces always prompt for saving
+ on exit if it would modify the contents of the ``.config`` file).
+
+ Due to Kconfig semantics, simply loading an old ``.config`` file performs an
+ implicit ``olddefconfig``, so building will normally not be affected by
+ having an outdated configuration.
+
+Whenever ``.config`` is overwritten, the previous version of the file is saved
+to ``.config.old`` (or, more generally, to ``$KCONFIG_CONFIG.old``).
+
+Using ``.config`` files as Make input
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``.config`` files use Make syntax and can be included directly in Makefiles to
+read configuration values from there. This is why ``n``-valued
+``bool``/``tristate`` values are written out as ``# CONFIG_FOO is not set`` (a
+Make comment) in ``.config``, allowing them to be tested with ``ifdef`` in
+Make.
+
+If you make use of this, you might want to pass ``--config-out <filename>`` to
+``genconfig`` and include the configuration file it generates instead of
+including ``.config`` directly. This has the advantage that the generated
+configuration file will always be a "full" configuration file, even if
+``.config`` is outdated. Otherwise, it might be necessary to run
+``old(def)config`` or ``menuconfig``/``guiconfig`` before rebuilding with an
+outdated ``.config``.
+
+If you use ``--sync-deps`` to generate incremental build information, you can
+include ``deps/auto.conf`` instead, which is also a full configuration file.
+
+Useful helper macros
+~~~~~~~~~~~~~~~~~~~~
+
+The `include/linux/kconfig.h
+<https://github.com/torvalds/linux/blob/master/include/linux/kconfig.h>`_
+header in the Linux kernel defines some useful helper macros for testing
+Kconfig configuration values.
+
+``IS_ENABLED()`` is generally useful, allowing configuration values to be
+tested in ``if`` statements with no runtime overhead.
+
+Incremental building
+~~~~~~~~~~~~~~~~~~~~
+
+See the docstring for ``Kconfig.sync_deps()`` in `kconfiglib.py
+<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_ for hints
+on implementing incremental builds (rebuilding just source files that reference
+changed configuration values).
+
+Running the ``scripts/basic/fixdep.c`` tool from the kernel on the output of
+``gcc -MD <source file>`` might give you an idea of how it all fits together.
+
+Library documentation
+---------------------
+
+Kconfiglib comes with extensive documentation in the form of docstrings. To view it, run e.g.
+the following command:
+
+.. code:: sh
+
+ $ pydoc(3) kconfiglib
+
+For HTML output, add ``-w``:
+
+.. code:: sh
+
+ $ pydoc(3) -w kconfiglib
+
+This will also work after installing Kconfiglib with ``pip(3)``.
+
+Documentation for other modules can be viewed in the same way (though a plain
+``--help`` will work when they're run as executables):
+
+.. code:: sh
+
+ $ pydoc(3) menuconfig/guiconfig/...
+
+A good starting point for learning the library is to read the module docstring
+(which you could also just read directly at the beginning of `kconfiglib.py
+<https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_). It
+gives an introduction to symbol values, the menu tree, and expressions.
+
+After reading the module docstring, a good next step is to read the ``Kconfig``
+class documentation, and then the documentation for the ``Symbol``, ``Choice``,
+and ``MenuNode`` classes.
+
+Please tell me if something is unclear or can be explained better.
+
+Library features
+----------------
+
+Kconfiglib can do the following, among other things:
+
+- **Programmatically get and set symbol values**
+
+ See `allnoconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/allnoconfig.py>`_ and
+ `allyesconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_,
+ which are automatically verified to produce identical output to the standard
+ ``make allnoconfig`` and ``make allyesconfig``.
+
+- **Read and write .config and defconfig files**
+
+ The generated ``.config`` and ``defconfig`` (minimal configuration) files are
+ character-for-character identical to what the C implementation would generate
+ (except for the header comment). The test suite relies on this, as it
+ compares the generated files.
+
+- **Write C headers**
+
+ The generated headers use the same format as ``include/generated/autoconf.h``
+ from the Linux kernel. Output for symbols appears in the order that they're
+ defined, unlike in the C tools (where the order depends on the hash table
+ implementation).
+
+- **Implement incremental builds**
+
+ This uses the same scheme as the ``include/config`` directory in the kernel:
+ Symbols are translated into files that are touched when the symbol's value
+ changes between builds, which can be used to avoid having to do a full
+ rebuild whenever the configuration is changed.
+
+ See the ``sync_deps()`` function for more information.
+
+- **Inspect symbols**
+
+ Printing a symbol or other item (which calls ``__str__()``) returns its
+ definition in Kconfig format. This also works for symbols defined in multiple
+ locations.
+
+ A helpful ``__repr__()`` is on all objects too.
+
+ All ``__str__()`` and ``__repr__()`` methods are deliberately implemented
+ with just public APIs, so all symbol information can be fetched separately as
+ well.
+
+- **Inspect expressions**
+
+ Expressions use a simple tuple-based format that can be processed manually
+ if needed. Expression printing and evaluation functions are provided,
+ implemented with public APIs.
+
+- **Inspect the menu tree**
+
+ The underlying menu tree is exposed, including submenus created implicitly
+ from symbols depending on preceding symbols. This can be used e.g. to
+ implement menuconfig-like functionality.
+
+ See `menuconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_/`guiconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ and the
+ minimalistic `menuconfig_example.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_
+ example.
+
+Kconfig extensions
+~~~~~~~~~~~~~~~~~~
+
+The following Kconfig extensions are available:
+
+- ``source`` supports glob patterns and includes each matching file. A pattern
+ is required to match at least one file.
+
+ A separate ``osource`` statement is available for cases where it's okay for
+ the pattern to match no files (in which case ``osource`` turns into a no-op).
+
+- A relative ``source`` statement (``rsource``) is available, where file paths
+ are specified relative to the directory of the current Kconfig file. An
+ ``orsource`` statement is available as well, analogous to ``osource``.
+
+- Preprocessor user functions can be defined in Python, which makes it simple
+ to integrate information from existing Python tools into Kconfig (e.g. to
+ have Kconfig symbols depend on hardware information stored in some other
+ format).
+
+ See the *Kconfig extensions* section in the
+ `kconfiglib.py <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_
+ module docstring for more information.
+
+- ``def_int``, ``def_hex``, and ``def_string`` are available in addition to
+ ``def_bool`` and ``def_tristate``, allowing ``int``, ``hex``, and ``string``
+ symbols to be given a type and a default at the same time.
+
+ These can be useful in projects that make use of symbols defined in multiple
+ locations, and remove some Kconfig inconsistency.
+
+- Environment variables are expanded directly in e.g. ``source`` and
+ ``mainmenu`` statements, meaning ``option env`` symbols are redundant.
+
+ This is the standard behavior with the new `Kconfig preprocessor
+ <https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.rst>`__,
+ which Kconfiglib implements.
+
+ ``option env`` symbols are accepted but ignored, which leads the caveat that
+ they must have the same name as the environment variables they reference
+ (Kconfiglib warns if the names differ). This keeps Kconfiglib compatible with
+ older Linux kernels, where the name of the ``option env`` symbol always
+ matched the environment variable. Compatibility with older Linux kernels is
+ the main reason ``option env`` is still supported.
+
+ The C tools have dropped support for ``option env``.
+
+- Two extra optional warnings can be enabled by setting environment variables,
+ covering cases that are easily missed when making changes to Kconfig files:
+
+ * ``KCONFIG_WARN_UNDEF``: If set to ``y``, warnings will be generated for all
+ references to undefined symbols within Kconfig files. The only gotcha is
+ that all hex literals must be prefixed with ``0x`` or ``0X``, to make it
+ possible to distinguish them from symbol references.
+
+ Some projects (e.g. the Linux kernel) use multiple Kconfig trees with many
+ shared Kconfig files, leading to some safe undefined symbol references.
+ ``KCONFIG_WARN_UNDEF`` is useful in projects that only have a single
+ Kconfig tree though.
+
+ ``KCONFIG_STRICT`` is an older alias for this environment variable,
+ supported for backwards compatibility.
+
+ * ``KCONFIG_WARN_UNDEF_ASSIGN``: If set to ``y``, warnings will be generated
+ for all assignments to undefined symbols within ``.config`` files. By
+ default, no such warnings are generated.
+
+ This warning can also be enabled/disabled by setting
+ ``Kconfig.warn_assign_undef`` to ``True``/``False``.
+
+Other features
+--------------
+
+- **Single-file implementation**
+
+ The entire library is contained in `kconfiglib.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/kconfiglib.py>`_.
+
+ The tools implemented on top of it are one file each.
+
+- **Robust and highly compatible with the C Kconfig tools**
+
+ The `test suite <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
+ automatically compares output from Kconfiglib and the C tools
+ by diffing the generated ``.config`` files for the real kernel Kconfig and
+ defconfig files, for all ARCHes.
+
+ This currently involves comparing the output for 36 ARCHes and 498 defconfig
+ files (or over 18000 ARCH/defconfig combinations in "obsessive" test suite
+ mode). All tests are expected to pass.
+
+ A comprehensive suite of selftests is included as well.
+
+- **Not horribly slow despite being a pure Python implementation**
+
+ The `allyesconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/allyesconfig.py>`_
+ script currently runs in about 1.3 seconds on the Linux kernel on a Core i7
+ 2600K (with a warm file cache), including the ``make`` overhead from ``make
+ scriptconfig``. Note that the Linux kernel Kconfigs are absolutely massive
+ (over 14k symbols for x86) compared to most projects, and also have overhead
+ from running shell commands via the Kconfig preprocessor.
+
+ Kconfiglib is especially speedy in cases where multiple ``.config`` files
+ need to be processed, because the ``Kconfig`` files will only need to be parsed
+ once.
+
+ For long-running jobs, `PyPy <https://pypy.org/>`_ gives a big performance
+ boost. CPython is faster for short-running jobs as PyPy needs some time to
+ warm up.
+
+ Kconfiglib also works well with the
+ `multiprocessing <https://docs.python.org/3/library/multiprocessing.html>`_
+ module. No global state is kept.
+
+- **Generates more warnings than the C implementation**
+
+ Generates the same warnings as the C implementation, plus additional ones.
+ Also detects dependency and ``source`` loops.
+
+ All warnings point out the location(s) in the ``Kconfig`` files where a
+ symbol is defined, where applicable.
+
+- **Unicode support**
+
+ Unicode characters in string literals in ``Kconfig`` and ``.config`` files are
+ correctly handled. This support mostly comes for free from Python.
+
+- **Windows support**
+
+ Nothing Linux-specific is used. Universal newlines mode is used for both
+ Python 2 and Python 3.
+
+ The `Zephyr <https://www.zephyrproject.org/>`_ project uses Kconfiglib to
+ generate ``.config`` files and C headers on Linux as well as Windows.
+
+- **Internals that (mostly) mirror the C implementation**
+
+ While being simpler to understand and tweak.
+
+Menuconfig interfaces
+---------------------
+
+Three configuration interfaces are currently available:
+
+- `menuconfig.py <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_
+ is a terminal-based configuration interface implemented using the standard
+ Python ``curses`` module. ``xconfig`` features like showing invisible symbols and
+ showing symbol names are included, and it's possible to jump directly to a symbol
+ in the menu tree (even if it's currently invisible).
+
+ .. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/menuconfig.gif
+
+ *There is now also a show-help mode that shows the help text of the currently
+ selected symbol in the help window at the bottom.*
+
+ Starting with Kconfiglib 12.2.0, ``menuconfig.py`` runs under both Python 2
+ and Python 3 (previously, it only ran under Python 3, so this was a
+ backport). Running it under Python 3 provides better support for Unicode text
+ entry (``get_wch()`` is not available in the ``curses`` module on Python 2).
+
+ There are no third-party dependencies on \*nix. On Windows,
+ the ``curses`` modules is not available by default, but support
+ can be added by installing the ``windows-curses`` package:
+
+ .. code-block:: shell
+
+ $ pip install windows-curses
+
+ This uses wheels built from `this repository
+ <https://github.com/zephyrproject-rtos/windows-curses>`_, which is in turn
+ based on Christoph Gohlke's `Python Extension Packages for Windows
+ <https://www.lfd.uci.edu/~gohlke/pythonlibs/#curses>`_.
+
+ See the docstring at the top of `menuconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py>`_ for
+ more information about the terminal menuconfig implementation.
+
+- `guiconfig.py
+ <https://github.com/ulfalizer/Kconfiglib/blob/master/guiconfig.py>`_ is a
+ graphical configuration interface written in `Tkinter
+ <https://docs.python.org/3/library/tkinter.html>`_. Like ``menuconfig.py``,
+ it supports showing all symbols (with invisible symbols in red) and jumping
+ directly to symbols. Symbol values can also be changed directly from the
+ jump-to dialog.
+
+ When single-menu mode is enabled, a single menu is shown at a time, like in
+ the terminal menuconfig. Only this mode distinguishes between symbols defined
+ with ``config`` and symbols defined with ``menuconfig``.
+
+ ``guiconfig.py`` has been tested on X11, Windows, and macOS, and is
+ compatible with both Python 2 and Python 3.
+
+ Despite being part of the Python standard library, ``tkinter`` often isn't
+ included by default in Python installations on Linux. These commands will
+ install it on a few different distributions:
+
+ - Ubuntu: ``sudo apt install python-tk``/``sudo apt install python3-tk``
+
+ - Fedora: ``dnf install python2-tkinter``/``dnf install python3-tkinter``
+
+ - Arch: ``sudo pacman -S tk``
+
+ - Clear Linux: ``sudo swupd bundle-add python3-tcl``
+
+ Screenshot below, with show-all mode enabled and the jump-to dialog open:
+
+ .. image:: https://raw.githubusercontent.com/ulfalizer/Kconfiglib/screenshots/screenshots/guiconfig.png
+
+ To avoid having to carry around a bunch of GIFs, the image data is embedded
+ in ``guiconfig.py``. To use separate GIF files instead, change
+ ``_USE_EMBEDDED_IMAGES`` to ``False`` in ``guiconfig.py``. The image files
+ can be found in the `screenshots
+ <https://github.com/ulfalizer/Kconfiglib/tree/screenshots/guiconfig>`_
+ branch.
+
+ I did my best with the images, but some are definitely only art adjacent.
+ Touch-ups are welcome. :)
+
+- `pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_, built by `RomaVis
+ <https://github.com/RomaVis>`_, is an older portable Python 2/3 TkInter
+ menuconfig implementation.
+
+ Screenshot below:
+
+ .. image:: https://raw.githubusercontent.com/RomaVis/pymenuconfig/master/screenshot.PNG
+
+ While working on the terminal menuconfig implementation, I added a few APIs
+ to Kconfiglib that turned out to be handy. ``pymenuconfig`` predates
+ ``menuconfig.py`` and ``guiconfig.py``, and so didn't have them available.
+ Blame me for any workarounds.
+
+Examples
+--------
+
+Example scripts
+~~~~~~~~~~~~~~~
+
+The `examples/ <https://github.com/ulfalizer/Kconfiglib/blob/master/examples>`_ directory contains some simple example scripts. Among these are the following ones. Make sure you run them with the latest version of Kconfiglib, as they might make use of newly added features.
+
+- `eval_expr.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/eval_expr.py>`_ evaluates an expression in the context of a configuration.
+
+- `find_symbol.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/find_symbol.py>`_ searches through expressions to find references to a symbol, also printing a "backtrace" with parents for each reference found.
+
+- `help_grep.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/help_grep.py>`_ searches for a string in all help texts.
+
+- `print_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_tree.py>`_ prints a tree of all configuration items.
+
+- `print_config_tree.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/print_config_tree.py>`_ is similar to ``print_tree.py``, but dumps the tree as it would appear in ``menuconfig``, including values. This can be handy for visually diffing between ``.config`` files and different versions of ``Kconfig`` files.
+
+- `list_undefined.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/list_undefined.py>`_ finds references to symbols that are not defined by any architecture in the Linux kernel.
+
+- `merge_config.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/merge_config.py>`_ merges configuration fragments to produce a complete .config, similarly to ``scripts/kconfig/merge_config.sh`` from the kernel.
+
+- `menuconfig_example.py <https://github.com/ulfalizer/Kconfiglib/blob/master/examples/menuconfig_example.py>`_ implements a configuration interface that uses notation similar to ``make menuconfig``. It's deliberately kept as simple as possible to demonstrate just the core concepts.
+
+Real-world examples
+~~~~~~~~~~~~~~~~~~~
+
+- `kconfig.py
+ <https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/kconfig/kconfig.py>`_
+ from the `Zephyr <https://www.zephyrproject.org/>`_ project handles
+ ``.config`` and header file generation, also doing configuration fragment
+ merging
+
+- `genrest.py
+ <https://github.com/zephyrproject-rtos/zephyr/blob/master/doc/scripts/genrest.py>`_
+ generates a Kconfig symbol cross-reference, which can be viewed `here
+ <http://docs.zephyrproject.org/reference/kconfig/index.html>`__
+
+- `CMake and IDE integration
+ <https://github.com/espressif/esp-idf/tree/master/tools/kconfig_new>`_ from
+ the ESP-IDF project, via a configuration server program.
+
+- `A script for turning on USB-related options
+ <https://github.com/google/syzkaller/blob/master/dashboard/config/kconfiglib-merge-usb-configs.py>`_,
+ from the `syzkaller <https://github.com/google/syzkaller>`_ project.
+
+- `Various automated checks
+ <https://github.com/zephyrproject-rtos/ci-tools/blob/master/scripts/check_compliance.py>`_,
+ including a check for references to undefined Kconfig symbols in source code.
+ See the ``KconfigCheck`` class.
+
+- `Various utilities
+ <https://github.com/projectacrn/acrn-hypervisor/tree/master/scripts/kconfig>`_
+ from the `ACRN <https://projectacrn.org/>`_ project
+
+These use the older Kconfiglib 1 API, which was clunkier and not as general
+(functions instead of properties, no direct access to the menu structure or
+properties, uglier ``__str__()`` output):
+
+- `genboardscfg.py <http://git.denx.de/?p=u-boot.git;a=blob;f=tools/genboardscfg.py;hb=HEAD>`_ from `Das U-Boot <http://www.denx.de/wiki/U-Boot>`_ generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :).
+
+- `gen-manual-lists.py <https://git.busybox.net/buildroot/tree/support/scripts/gen-manual-lists.py?id=5676a2deea896f38123b99781da0a612865adeb0>`_ generated listings for an appendix in the `Buildroot <https://buildroot.org>`_ manual. (The listing has since been removed.)
+
+- `gen_kconfig_doc.py <https://github.com/espressif/esp-idf/blob/master/docs/gen-kconfig-doc.py>`_ from the `esp-idf <https://github.com/espressif/esp-idf>`_ project generates documentation from Kconfig files.
+
+- `SConf <https://github.com/CoryXie/SConf>`_ builds an interactive configuration interface (like ``menuconfig``) on top of Kconfiglib, for use e.g. with `SCons <scons.org>`_.
+
+- `kconfig-diff.py <https://gist.github.com/dubiousjim/5638961>`_ -- a script by `dubiousjim <https://github.com/dubiousjim>`_ that compares kernel configurations.
+
+- Originally, Kconfiglib was used in chapter 4 of my `master's thesis <http://liu.diva-portal.org/smash/get/diva2:473038/FULLTEXT01.pdf>`_ to automatically generate a "minimal" kernel for a given system. Parts of it bother me a bit now, but that's how it goes with old work.
+
+Sample ``make iscriptconfig`` session
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following log should give some idea of the functionality available in the API:
+
+.. code-block::
+
+ $ make iscriptconfig
+ A Kconfig instance 'kconf' for the architecture x86 has been created.
+ >>> kconf # Calls Kconfig.__repr__()
+ <configuration with 13711 symbols, main menu prompt "Linux/x86 4.14.0-rc7 Kernel Configuration", srctree ".", config symbol prefix "CONFIG_", warnings enabled, undef. symbol assignment warnings disabled>
+ >>> kconf.mainmenu_text # Expanded main menu text
+ 'Linux/x86 4.14.0-rc7 Kernel Configuration'
+ >>> kconf.top_node # The implicit top-level menu
+ <menu node for menu, prompt "Linux/x86 4.14.0-rc7 Kernel Configuration" (visibility y), deps y, 'visible if' deps y, has child, Kconfig:5>
+ >>> kconf.top_node.list # First child menu node
+ <menu node for symbol SRCARCH, deps y, has next, Kconfig:7>
+ >>> print(kconf.top_node.list) # Calls MenuNode.__str__()
+ config SRCARCH
+ string
+ option env="SRCARCH"
+ default "x86"
+ >>> sym = kconf.top_node.list.next.item # Item contained in next menu node
+ >>> print(sym) # Calls Symbol.__str__()
+ config 64BIT
+ bool "64-bit kernel" if ARCH = "x86"
+ default ARCH != "i386"
+ help
+ Say yes to build a 64-bit kernel - formerly known as x86_64
+ Say no to build a 32-bit kernel - formerly known as i386
+ >>> sym # Calls Symbol.__repr__()
+ <symbol 64BIT, bool, "64-bit kernel", value y, visibility y, direct deps y, arch/x86/Kconfig:2>
+ >>> sym.assignable # Currently assignable values (0, 1, 2 = n, m, y)
+ (0, 2)
+ >>> sym.set_value(0) # Set it to n
+ True
+ >>> sym.tri_value # Check the new value
+ 0
+ >>> sym = kconf.syms["X86_MPPARSE"] # Look up symbol by name
+ >>> print(sym)
+ config X86_MPPARSE
+ bool "Enable MPS table" if (ACPI || SFI) && X86_LOCAL_APIC
+ default y if X86_LOCAL_APIC
+ help
+ For old smp systems that do not have proper acpi support. Newer systems
+ (esp with 64bit cpus) with acpi support, MADT and DSDT will override it
+ >>> default = sym.defaults[0] # Fetch its first default
+ >>> sym = default[1] # Fetch the default's condition (just a Symbol here)
+ >>> print(sym)
+ config X86_LOCAL_APIC
+ bool
+ default y
+ select IRQ_DOMAIN_HIERARCHY
+ select PCI_MSI_IRQ_DOMAIN if PCI_MSI
+ depends on X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI
+ >>> sym.nodes # Show the MenuNode(s) associated with it
+ [<menu node for symbol X86_LOCAL_APIC, deps n, has next, arch/x86/Kconfig:1015>]
+ >>> kconfiglib.expr_str(sym.defaults[0][1]) # Print the default's condition
+ 'X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_APIC || PCI_MSI'
+ >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it (0 = n)
+ 0
+ >>> kconf.syms["64BIT"].set_value(2)
+ True
+ >>> kconfiglib.expr_value(sym.defaults[0][1]) # Evaluate it again (2 = y)
+ 2
+ >>> kconf.write_config("myconfig") # Save a .config
+ >>> ^D
+ $ cat myconfig
+ # Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)
+ CONFIG_64BIT=y
+ CONFIG_X86_64=y
+ CONFIG_X86=y
+ CONFIG_INSTRUCTION_DECODER=y
+ CONFIG_OUTPUT_FORMAT="elf64-x86-64"
+ CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
+ CONFIG_LOCKDEP_SUPPORT=y
+ CONFIG_STACKTRACE_SUPPORT=y
+ CONFIG_MMU=y
+ ...
+
+Test suite
+----------
+
+The test suite is run with
+
+.. code::
+
+ $ python(3) Kconfiglib/testsuite.py
+
+`pypy <https://pypy.org/>`_ works too, and is much speedier for everything except ``allnoconfig.py``/``allnoconfig_simpler.py``/``allyesconfig.py``, where it doesn't have time to warm up since
+the scripts are run via ``make scriptconfig``.
+
+The test suite must be run from the top-level kernel directory. It requires that the
+Kconfiglib git repository has been cloned into it and that the makefile patch has been applied.
+
+To get rid of warnings generated for the kernel ``Kconfig`` files, add ``2>/dev/null`` to the command to
+discard ``stderr``.
+
+**NOTE: Forgetting to apply the Makefile patch will cause some tests that compare generated configurations to fail**
+
+**NOTE: The test suite overwrites .config in the kernel root, so make sure to back it up.**
+
+The test suite consists of a set of selftests and a set of compatibility tests that
+compare configurations generated by Kconfiglib with
+configurations generated by the C tools, for a number of cases. See
+`testsuite.py <https://github.com/ulfalizer/Kconfiglib/blob/master/testsuite.py>`_
+for the available options.
+
+The `tests/reltest <https://github.com/ulfalizer/Kconfiglib/blob/master/tests/reltest>`_ script runs the test suite
+and all the example scripts for both Python 2 and Python 3, verifying that everything works.
+
+Rarely, the output from the C tools is changed slightly (most recently due to a
+`change <https://www.spinics.net/lists/linux-kbuild/msg17074.html>`_ I added).
+If you get test suite failures, try running the test suite again against the
+`linux-next tree <https://www.kernel.org/doc/man-pages/linux-next.html>`_,
+which has all the latest changes. I will make it clear if any
+non-backwards-compatible changes appear.
+
+A lot of time is spent waiting around for ``make`` and the C utilities (which need to reparse all the
+Kconfig files for each defconfig test). Adding some multiprocessing to the test suite would make sense
+too.
+
+Notes
+-----
+
+* This is version 2 of Kconfiglib, which is not backwards-compatible with
+ Kconfiglib 1. A summary of changes between Kconfiglib 1 and Kconfiglib
+ 2 can be found `here
+ <https://github.com/ulfalizer/Kconfiglib/blob/screenshots/kconfiglib-2-changes.txt>`__.
+
+* I sometimes see people add custom output formats, which is pretty
+ straightforward to do (see the implementations of ``write_autoconf()`` and
+ ``write_config()`` for a template, and also the documentation of the
+ ``Symbol.config_string`` property). If you come up with something you think
+ might be useful to other people, I'm happy to take it in upstream. Batteries
+ included and all that.
+
+* Kconfiglib assumes the modules symbol is ``MODULES``, which is backwards-compatible.
+ A warning is printed by default if ``option modules`` is set on some other symbol.
+
+ Let me know if you need proper ``option modules`` support. It wouldn't be that
+ hard to add.
+
+Thanks
+------
+
+- To `RomaVis <https://github.com/RomaVis>`_, for making
+ `pymenuconfig <https://github.com/RomaVis/pymenuconfig>`_ and suggesting
+ the ``rsource`` keyword.
+
+- To `Mitja Horvat <https://github.com/pinkfluid>`_, for adding support
+ for user-defined styles to the terminal menuconfig.
+
+- To `Philip Craig <https://github.com/philipc>`_ for adding
+ support for the ``allnoconfig_y`` option and fixing an obscure issue
+ with ``comment``\s inside ``choice``\s (that didn't affect correctness but
+ made outputs differ). ``allnoconfig_y`` is used to force certain symbols
+ to ``y`` during ``make allnoconfig`` to improve coverage.
+
+License
+-------
+
+See `LICENSE.txt <https://github.com/ulfalizer/Kconfiglib/blob/master/LICENSE.txt>`_. SPDX license identifiers are used in the
+source code.
diff --git a/ext/Kconfiglib/alldefconfig.py b/ext/Kconfiglib/alldefconfig.py
new file mode 100755
index 0000000..56c4caa
--- /dev/null
+++ b/ext/Kconfiglib/alldefconfig.py
@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2018-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Writes a configuration file where all symbols are set to their their default
+values.
+
+The default output filename is '.config'. A different filename can be passed in
+the KCONFIG_CONFIG environment variable.
+
+Usage for the Linux kernel:
+
+ $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/alldefconfig.py
+"""
+import kconfiglib
+
+
+def main():
+ kconf = kconfiglib.standard_kconfig(__doc__)
+ kconf.load_allconfig("alldef.config")
+ print(kconf.write_config())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/allmodconfig.py b/ext/Kconfiglib/allmodconfig.py
new file mode 100755
index 0000000..bfb72b4
--- /dev/null
+++ b/ext/Kconfiglib/allmodconfig.py
@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2018-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Writes a configuration file where as many symbols as possible are set to 'm'.
+
+The default output filename is '.config'. A different filename can be passed
+in the KCONFIG_CONFIG environment variable.
+
+Usage for the Linux kernel:
+
+ $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allmodconfig.py
+"""
+import kconfiglib
+
+
+def main():
+ kconf = kconfiglib.standard_kconfig(__doc__)
+
+ # See allnoconfig.py
+ kconf.warn = False
+
+ for sym in kconf.unique_defined_syms:
+ if sym.orig_type == kconfiglib.BOOL:
+ # 'bool' choice symbols get their default value, as determined by
+ # e.g. 'default's on the choice
+ if not sym.choice:
+ # All other bool symbols get set to 'y', like for allyesconfig
+ sym.set_value(2)
+ elif sym.orig_type == kconfiglib.TRISTATE:
+ sym.set_value(1)
+
+ for choice in kconf.unique_choices:
+ choice.set_value(2 if choice.orig_type == kconfiglib.BOOL else 1)
+
+ kconf.warn = True
+
+ kconf.load_allconfig("allmod.config")
+
+ print(kconf.write_config())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/allnoconfig.py b/ext/Kconfiglib/allnoconfig.py
new file mode 100755
index 0000000..de90d8b
--- /dev/null
+++ b/ext/Kconfiglib/allnoconfig.py
@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2018-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Writes a configuration file where as many symbols as possible are set to 'n'.
+
+The default output filename is '.config'. A different filename can be passed
+in the KCONFIG_CONFIG environment variable.
+
+Usage for the Linux kernel:
+
+ $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allnoconfig.py
+"""
+
+# See examples/allnoconfig_walk.py for another way to implement this script
+
+import kconfiglib
+
+
+def main():
+ kconf = kconfiglib.standard_kconfig(__doc__)
+
+ # Avoid warnings that would otherwise get printed by Kconfiglib for the
+ # following:
+ #
+ # 1. Assigning a value to a symbol without a prompt, which never has any
+ # effect
+ #
+ # 2. Assigning values invalid for the type (only bool/tristate symbols
+ # accept 0/1/2, for n/m/y). The assignments will be ignored for other
+ # symbol types, which is what we want.
+ kconf.warn = False
+ for sym in kconf.unique_defined_syms:
+ sym.set_value(2 if sym.is_allnoconfig_y else 0)
+ kconf.warn = True
+
+ kconf.load_allconfig("allno.config")
+
+ print(kconf.write_config())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/allyesconfig.py b/ext/Kconfiglib/allyesconfig.py
new file mode 100755
index 0000000..90eb9b8
--- /dev/null
+++ b/ext/Kconfiglib/allyesconfig.py
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2018-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Writes a configuration file where as many symbols as possible are set to 'y'.
+
+The default output filename is '.config'. A different filename can be passed
+in the KCONFIG_CONFIG environment variable.
+
+Usage for the Linux kernel:
+
+ $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/allyesconfig.py
+"""
+import kconfiglib
+
+
+def main():
+ kconf = kconfiglib.standard_kconfig(__doc__)
+
+ # See allnoconfig.py
+ kconf.warn = False
+
+ # Try to set all symbols to 'y'. Dependencies might truncate the value down
+ # later, but this will at least give the highest possible value.
+ #
+ # Assigning 0/1/2 to non-bool/tristate symbols has no effect (int/hex
+ # symbols still take a string, because they preserve formatting).
+ for sym in kconf.unique_defined_syms:
+ # Set choice symbols to 'm'. This value will be ignored for choices in
+ # 'y' mode (the "normal" mode), which will instead just get their
+ # default selection, but will set all symbols in m-mode choices to 'm',
+ # which is as high as they can go.
+ #
+ # Here's a convoluted example of how you might get an m-mode choice
+ # even during allyesconfig:
+ #
+ # choice
+ # tristate "weird choice"
+ # depends on m
+ sym.set_value(1 if sym.choice else 2)
+
+ # Set all choices to the highest possible mode
+ for choice in kconf.unique_choices:
+ choice.set_value(2)
+
+ kconf.warn = True
+
+ kconf.load_allconfig("allyes.config")
+
+ print(kconf.write_config())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/defconfig.py b/ext/Kconfiglib/defconfig.py
new file mode 100755
index 0000000..b179273
--- /dev/null
+++ b/ext/Kconfiglib/defconfig.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Reads a specified configuration file, then writes a new configuration file.
+This can be used to initialize the configuration from e.g. an arch-specific
+configuration file. This input configuration file would usually be a minimal
+configuration file, as generated by e.g. savedefconfig.
+
+The default output filename is '.config'. A different filename can be passed in
+the KCONFIG_CONFIG environment variable.
+"""
+import argparse
+
+import kconfiglib
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ description=__doc__)
+
+ parser.add_argument(
+ "--kconfig",
+ default="Kconfig",
+ help="Top-level Kconfig file (default: Kconfig)")
+
+ parser.add_argument(
+ "config",
+ metavar="CONFIGURATION",
+ help="Input configuration file")
+
+ args = parser.parse_args()
+
+ kconf = kconfiglib.Kconfig(args.kconfig, suppress_traceback=True)
+ print(kconf.load_config(args.config))
+ print(kconf.write_config())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/examples/Kmenuconfig b/ext/Kconfiglib/examples/Kmenuconfig
new file mode 100644
index 0000000..f1cb67b
--- /dev/null
+++ b/ext/Kconfiglib/examples/Kmenuconfig
@@ -0,0 +1,102 @@
+mainmenu "Example Kconfig configuration"
+
+config MODULES
+ bool "Enable loadable module support"
+ option modules
+ default y
+
+menu "Bool and tristate symbols"
+
+config BOOL
+ bool "Bool symbol"
+ default y
+
+config BOOL_DEP
+ bool "Dependent bool symbol"
+ depends on BOOL
+
+# Mix it up a bit with an 'if' instead of a 'depends on'
+if BOOL
+
+config TRI_DEP
+ tristate "Dependent tristate symbol"
+ select SELECTED_BY_TRI_DEP
+ imply IMPLIED_BY_TRI_DEP
+
+endif
+
+config TWO_MENU_NODES
+ bool "First prompt"
+ depends on BOOL
+
+config TRI
+ tristate "Tristate symbol"
+
+config TWO_MENU_NODES
+ bool "Second prompt"
+
+comment "These are selected by TRI_DEP"
+
+config SELECTED_BY_TRI_DEP
+ tristate "Tristate selected by TRI_DEP"
+
+config IMPLIED_BY_TRI_DEP
+ tristate "Tristate implied by TRI_DEP"
+
+endmenu
+
+
+menu "String, int, and hex symbols"
+
+config STRING
+ string "String symbol"
+ default "foo"
+
+config INT
+ int "Int symbol"
+ default 747
+
+config HEX
+ hex "Hex symbol"
+ default 0xABC
+
+endmenu
+
+
+menu "Various choices"
+
+choice BOOL_CHOICE
+ bool "Bool choice"
+
+config BOOL_CHOICE_SYM_1
+ bool "Bool choice sym 1"
+
+config BOOL_CHOICE_SYM_2
+ bool "Bool choice sym 2"
+
+endchoice
+
+choice TRI_CHOICE
+ tristate "Tristate choice"
+
+config TRI_CHOICE_SYM_1
+ tristate "Tristate choice sym 1"
+
+config TRI_CHOICE_SYM_2
+ tristate "Tristate choice sym 2"
+
+endchoice
+
+choice OPT_BOOL_CHOICE
+ bool "Optional bool choice"
+ optional
+
+config OPT_BOOL_CHOICE_SYM_1
+ bool "Optional bool choice sym 1"
+
+config OPT_BOOL_CHOICE_SYM_2
+ bool "Optional bool choice sym 2"
+
+endchoice
+
+endmenu
diff --git a/ext/Kconfiglib/examples/allnoconfig_walk.py b/ext/Kconfiglib/examples/allnoconfig_walk.py
new file mode 100644
index 0000000..5a8cc23
--- /dev/null
+++ b/ext/Kconfiglib/examples/allnoconfig_walk.py
@@ -0,0 +1,66 @@
+# This is tree-walking version of allnoconfig.py, for demonstration purposes.
+# Verified by the test suite to generate identical output to 'make allnoconfig'
+# for all ARCHes.
+#
+# Note: A more practical version would use Kconfig.node_iter(). The manual tree
+# walking is for demonstration purposes.
+#
+# Usage for the Linux kernel:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/allnoconfig_walk.py
+
+import sys
+
+from kconfiglib import Kconfig, Symbol
+
+
+def do_allnoconfig(node):
+ global changed
+
+ # Walk the tree of menu nodes. You can imagine this as going down/into menu
+ # entries in the menuconfig interface, setting each to n (or the lowest
+ # assignable value).
+
+ while node:
+ if isinstance(node.item, Symbol):
+ sym = node.item
+
+ # Is the symbol a non-allnoconfig_y symbol that can be set to a
+ # lower value than its current value?
+ if (not sym.is_allnoconfig_y and
+ sym.assignable and
+ sym.assignable[0] < sym.tri_value):
+
+ # Yup, lower it
+ sym.set_value(sym.assignable[0])
+ changed = True
+
+ # Recursively lower children
+ if node.list:
+ do_allnoconfig(node.list)
+
+ node = node.next
+
+
+# Parse the Kconfig files
+kconf = Kconfig(sys.argv[1])
+
+# Do an initial pass to set 'option allnoconfig_y' symbols to y
+for sym in kconf.unique_defined_syms:
+ if sym.is_allnoconfig_y:
+ sym.set_value(2)
+
+while True:
+ # Changing later symbols in the configuration can sometimes allow earlier
+ # symbols to be lowered, e.g. if a later symbol 'select's an earlier
+ # symbol. To handle such situations, we do additional passes over the tree
+ # until we're no longer able to change the value of any symbol in a pass.
+ changed = False
+
+ do_allnoconfig(kconf.top_node)
+
+ # Did the pass change any symbols?
+ if not changed:
+ break
+
+print(kconf.write_config())
diff --git a/ext/Kconfiglib/examples/defconfig_oldconfig.py b/ext/Kconfiglib/examples/defconfig_oldconfig.py
new file mode 100644
index 0000000..68336c6
--- /dev/null
+++ b/ext/Kconfiglib/examples/defconfig_oldconfig.py
@@ -0,0 +1,39 @@
+# Produces exactly the same output as the following script:
+#
+# make defconfig
+# echo CONFIG_ETHERNET=n >> .config
+# make oldconfig
+# echo CONFIG_ETHERNET=y >> .config
+# yes n | make oldconfig
+#
+# This came up in https://github.com/ulfalizer/Kconfiglib/issues/15.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/defconfig_oldconfig.py
+
+import sys
+
+import kconfiglib
+
+
+kconf = kconfiglib.Kconfig(sys.argv[1])
+
+# Mirrors defconfig
+kconf.load_config("arch/x86/configs/x86_64_defconfig")
+kconf.write_config()
+
+# Mirrors the first oldconfig
+kconf.load_config()
+kconf.syms["ETHERNET"].set_value(0)
+kconf.write_config()
+
+# Mirrors the second oldconfig
+kconf.load_config()
+kconf.syms["ETHERNET"].set_value(2)
+for s in kconf.unique_defined_syms:
+ if s.user_value is None and 0 in s.assignable:
+ s.set_value(0)
+
+# Write the final configuration
+print(kconf.write_config())
diff --git a/ext/Kconfiglib/examples/dumpvars.py b/ext/Kconfiglib/examples/dumpvars.py
new file mode 100644
index 0000000..0f8ab43
--- /dev/null
+++ b/ext/Kconfiglib/examples/dumpvars.py
@@ -0,0 +1,15 @@
+# Prints all (set) environment variables referenced in the Kconfig files
+# together with their values, as a list of assignments.
+#
+# Note: This only works for environment variables referenced via the $(FOO)
+# preprocessor syntax. The older $FOO syntax is maintained for backwards
+# compatibility.
+
+import os
+import sys
+
+import kconfiglib
+
+
+print(" ".join("{}='{}'".format(var, os.environ[var])
+ for var in kconfiglib.Kconfig(sys.argv[1]).env_vars))
diff --git a/ext/Kconfiglib/examples/eval_expr.py b/ext/Kconfiglib/examples/eval_expr.py
new file mode 100644
index 0000000..23eedb4
--- /dev/null
+++ b/ext/Kconfiglib/examples/eval_expr.py
@@ -0,0 +1,24 @@
+# Evaluates an expression (e.g. "X86_64 || (X86_32 && X86_LOCAL_APIC)") in the
+# context of a configuration. Note that this always yields a tristate value (n,
+# m, or y).
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/eval_expr.py SCRIPT_ARG=<expr>
+
+import sys
+
+import kconfiglib
+
+
+if len(sys.argv) < 3:
+ sys.exit("Pass the expression to evaluate with SCRIPT_ARG=<expression>")
+
+kconf = kconfiglib.Kconfig(sys.argv[1])
+expr = sys.argv[2]
+
+# Enable modules so that m doesn't get demoted to n
+kconf.modules.set_value(2)
+
+print("the expression '{}' evaluates to {}"
+ .format(expr, kconf.eval_string(expr)))
diff --git a/ext/Kconfiglib/examples/find_symbol.py b/ext/Kconfiglib/examples/find_symbol.py
new file mode 100644
index 0000000..f747103
--- /dev/null
+++ b/ext/Kconfiglib/examples/find_symbol.py
@@ -0,0 +1,112 @@
+# Prints all menu nodes that reference a given symbol any of their properties
+# or property conditions, along with their parent menu nodes.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/find_symbol.py SCRIPT_ARG=<name>
+#
+# Example output for SCRIPT_ARG=X86:
+#
+# Found 470 locations that reference X86:
+#
+# ========== Location 1 (init/Kconfig:1108) ==========
+#
+# config SGETMASK_SYSCALL
+# bool
+# prompt "sgetmask/ssetmask syscalls support" if EXPERT
+# default PARISC || M68K || PPC || MIPS || X86 || SPARC || MICROBLAZE || SUPERH
+# help
+# sys_sgetmask and sys_ssetmask are obsolete system calls
+# no longer supported in libc but still enabled by default in some
+# architectures.
+#
+# If unsure, leave the default option here.
+#
+# ---------- Parent 1 (init/Kconfig:1077) ----------
+#
+# menuconfig EXPERT
+# bool
+# prompt "Configure standard kernel features (expert users)"
+# select DEBUG_KERNEL
+# help
+# This option allows certain base kernel options and settings
+# to be disabled or tweaked. This is for specialized
+# environments which can tolerate a "non-standard" kernel.
+# Only use this if you really know what you are doing.
+#
+# ---------- Parent 2 (init/Kconfig:39) ----------
+#
+# menu "General setup"
+#
+# ========== Location 2 (arch/Kconfig:29) ==========
+#
+# config OPROFILE_EVENT_MULTIPLEX
+# bool
+# prompt "OProfile multiplexing support (EXPERIMENTAL)"
+# default "n"
+# depends on OPROFILE && X86
+# help
+# The number of hardware counters is limited. The multiplexing
+# feature enables OProfile to gather more events than counters
+# are provided by the hardware. This is realized by switching
+# between events at a user specified time interval.
+#
+# If unsure, say N.
+#
+# ---------- Parent 1 (arch/Kconfig:16) ----------
+#
+# config OPROFILE
+# tristate
+# prompt "OProfile system profiling"
+# select RING_BUFFER
+# select RING_BUFFER_ALLOW_SWAP
+# depends on PROFILING && HAVE_OPROFILE
+# help
+# OProfile is a profiling system capable of profiling the
+# whole system, include the kernel, kernel modules, libraries,
+# and applications.
+#
+# If unsure, say N.
+#
+# ---------- Parent 2 (init/Kconfig:39) ----------
+#
+# menu "General setup"
+#
+# ... (tons more)
+
+import sys
+
+import kconfiglib
+
+
+if len(sys.argv) < 3:
+ sys.exit('Pass symbol name (without "CONFIG_" prefix) with SCRIPT_ARG=<name>')
+
+kconf = kconfiglib.Kconfig(sys.argv[1])
+sym_name = sys.argv[2]
+if sym_name not in kconf.syms:
+ print("No symbol {} exists in the configuration".format(sym_name))
+ sys.exit(0)
+
+referencing = [node for node in kconf.node_iter()
+ if kconf.syms[sym_name] in node.referenced]
+if not referencing:
+ print("No references to {} found".format(sym_name))
+ sys.exit(0)
+
+print("Found {} locations that reference {}:\n"
+ .format(len(referencing), sym_name))
+
+for i, node in enumerate(referencing, 1):
+ print("========== Location {} ({}:{}) ==========\n\n{}"
+ .format(i, node.filename, node.linenr, node))
+
+ # Print the parents of the menu node too
+
+ node = node.parent
+ parent_i = 1
+ while node is not kconf.top_node:
+ print("---------- Parent {} ({}:{}) ----------\n\n{}"
+ .format(parent_i, node.filename, node.linenr, node))
+ node = node.parent
+ parent_i += 1
diff --git a/ext/Kconfiglib/examples/help_grep.py b/ext/Kconfiglib/examples/help_grep.py
new file mode 100644
index 0000000..157d8f2
--- /dev/null
+++ b/ext/Kconfiglib/examples/help_grep.py
@@ -0,0 +1,64 @@
+# Does a case-insensitive search for a regular expression in the help texts of
+# symbols and choices and the prompts of menus and comments. Prints the
+# matching items together with their locations and the matching text.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/help_grep.py SCRIPT_ARG=<regex>
+#
+# Shortened example output for SCRIPT_ARG=general:
+#
+# menu "General setup"
+# location: init/Kconfig:39
+#
+# config SYSVIPC
+# bool
+# prompt "System V IPC"
+# help
+# ...
+# exchange information. It is generally considered to be a good thing,
+# ...
+#
+# location: init/Kconfig:233
+#
+# config BSD_PROCESS_ACCT
+# bool
+# prompt "BSD Process Accounting" if MULTIUSER
+# help
+# ...
+# information. This is generally a good idea, so say Y.
+#
+# location: init/Kconfig:403
+#
+# ...
+
+
+import re
+import sys
+
+from kconfiglib import Kconfig, Symbol, Choice, MENU, COMMENT
+
+
+if len(sys.argv) < 3:
+ sys.exit("Pass the regex with SCRIPT_ARG=<regex>")
+
+search = re.compile(sys.argv[2], re.IGNORECASE).search
+
+for node in Kconfig(sys.argv[1]).node_iter():
+ match = False
+
+ if isinstance(node.item, (Symbol, Choice)) and \
+ node.help is not None and search(node.help):
+ print(node.item)
+ match = True
+
+ elif node.item == MENU and search(node.prompt[0]):
+ print('menu "{}"'.format(node.prompt[0]))
+ match = True
+
+ elif node.item == COMMENT and search(node.prompt[0]):
+ print('comment "{}"'.format(node.prompt[0]))
+ match = True
+
+ if match:
+ print("location: {}:{}\n".format(node.filename, node.linenr))
diff --git a/ext/Kconfiglib/examples/list_undefined.py b/ext/Kconfiglib/examples/list_undefined.py
new file mode 100644
index 0000000..4a3bc9b
--- /dev/null
+++ b/ext/Kconfiglib/examples/list_undefined.py
@@ -0,0 +1,156 @@
+# Prints a list of symbols that are referenced in the Kconfig files of some
+# architecture but not defined by the Kconfig files of any architecture.
+#
+# A Kconfig file might be shared between many architectures and legitimately
+# reference undefined symbols for some of them, but if no architecture defines
+# the symbol, it usually indicates a problem or potential cleanup.
+#
+# This script could be sped up a lot if needed. See the comment near the
+# referencing_nodes() call.
+#
+# Run with the following command in the kernel root:
+#
+# $ python(3) Kconfiglib/examples/list_undefined.py
+#
+# Example output:
+#
+# Registering defined and undefined symbols for all arches
+# Processing mips
+# Processing ia64
+# Processing metag
+# ...
+#
+# Finding references to each undefined symbol
+# Processing mips
+# Processing ia64
+# Processing metag
+# ...
+#
+# The following globally undefined symbols were found, listed here
+# together with the locations of the items that reference them.
+# References might come from enclosing menus and ifs.
+#
+# ARM_ERRATA_753970: arch/arm/mach-mvebu/Kconfig:56, arch/arm/mach-mvebu/Kconfig:39
+# SUNXI_CCU_MP: drivers/clk/sunxi-ng/Kconfig:14
+# SUNXI_CCU_DIV: drivers/clk/sunxi-ng/Kconfig:14
+# AC97: sound/ac97/Kconfig:6
+# ...
+
+import os
+import subprocess
+
+from kconfiglib import Kconfig
+
+
+# Referenced inside the Kconfig files
+os.environ["KERNELVERSION"] = str(
+ subprocess.check_output(("make", "kernelversion")).decode("utf-8").rstrip()
+)
+
+
+def all_arch_srcarch_pairs():
+ """
+ Generates all valid (ARCH, SRCARCH) tuples for the kernel, corresponding to
+ different architectures. SRCARCH holds the arch/ subdirectory.
+ """
+ for srcarch in os.listdir("arch"):
+ # Each subdirectory of arch/ containing a Kconfig file corresponds to
+ # an architecture
+ if os.path.exists(os.path.join("arch", srcarch, "Kconfig")):
+ yield (srcarch, srcarch)
+
+ # Some architectures define additional ARCH settings with ARCH != SRCARCH
+ # (search for "Additional ARCH settings for" in the top-level Makefile)
+
+ yield ("i386", "x86")
+ yield ("x86_64", "x86")
+
+ yield ("sparc32", "sparc")
+ yield ("sparc64", "sparc")
+
+ yield ("sh64", "sh")
+
+ yield ("um", "um")
+
+
+def all_arch_srcarch_kconfigs():
+ """
+ Generates Kconfig instances for all the architectures in the kernel
+ """
+
+ os.environ["srctree"] = "."
+ os.environ["HOSTCC"] = "gcc"
+ os.environ["HOSTCXX"] = "g++"
+ os.environ["CC"] = "gcc"
+ os.environ["LD"] = "ld"
+
+ for arch, srcarch in all_arch_srcarch_pairs():
+ print(" Processing " + arch)
+
+ os.environ["ARCH"] = arch
+ os.environ["SRCARCH"] = srcarch
+
+ # um (User Mode Linux) uses a different base Kconfig file
+ yield Kconfig("Kconfig" if arch != "um" else "arch/x86/um/Kconfig",
+ warn=False)
+
+
+print("Registering defined and undefined symbols for all arches")
+
+# Sets holding the names of all defined and undefined symbols, for all
+# architectures
+defined = set()
+undefined = set()
+
+for kconf in all_arch_srcarch_kconfigs():
+ for name, sym in kconf.syms.items():
+ if sym.nodes:
+ # If the symbol has a menu node, it is defined
+ defined.add(name)
+ else:
+ # Undefined symbol. We skip some of the uninteresting ones.
+
+ # Due to how Kconfig works, integer literals show up as symbols
+ # (from e.g. 'default 1'). Skip those.
+ try:
+ int(name, 0)
+ continue
+ except ValueError:
+ # Interesting undefined symbol
+ undefined.add(name)
+
+
+print("\nFinding references to each undefined symbol")
+
+def referencing_nodes(kconf, name):
+ # Returns a list of all menu nodes that reference a symbol named 'name' in
+ # any of their properties or property conditions
+ res = []
+
+ for node in kconf.node_iter():
+ for ref in node.referenced:
+ if ref.name == name:
+ res.append(node)
+
+ return res
+
+
+# Maps each globally undefined symbol to the menu nodes that reference it
+undef_sym_refs = [(name, set()) for name in undefined - defined]
+
+for kconf in all_arch_srcarch_kconfigs():
+ for name, refs in undef_sym_refs:
+ # This means that we search the entire configuration tree for each
+ # undefined symbol, which is terribly inefficient. We could speed
+ # things up by tweaking referencing_nodes() to compare each symbol to
+ # multiple symbols while walking the configuration tree.
+ for node in referencing_nodes(kconf, name):
+ refs.add("{}:{}".format(node.filename, node.linenr))
+
+
+print("\nThe following globally undefined symbols were found, listed here\n"
+ "together with the locations of the items that reference them.\n"
+ "References might come from enclosing menus and ifs.\n")
+
+for name, refs in undef_sym_refs:
+ print(" {}: {}".format(name, ", ".join(refs)))
diff --git a/ext/Kconfiglib/examples/menuconfig_example.py b/ext/Kconfiglib/examples/menuconfig_example.py
new file mode 100755
index 0000000..606f756
--- /dev/null
+++ b/ext/Kconfiglib/examples/menuconfig_example.py
@@ -0,0 +1,341 @@
+#!/usr/bin/env python3
+
+# Implements a simple configuration interface on top of Kconfiglib to
+# demonstrate concepts for building a menuconfig-like. Emulates how the
+# standard menuconfig prints menu entries.
+#
+# Always displays the entire Kconfig tree to keep things as simple as possible
+# (all symbols, choices, menus, and comments).
+#
+# Usage:
+#
+# $ python(3) Kconfiglib/examples/menuconfig.py <Kconfig file>
+#
+# A sample Kconfig is available in Kconfiglib/examples/Kmenuconfig.
+#
+# Here's a notation guide. The notation matches the one used by menuconfig
+# (scripts/kconfig/mconf):
+#
+# [ ] prompt - Bool
+# < > prompt - Tristate
+# {M} prompt - Tristate selected to m. Can only be set to m or y.
+# -*- prompt - Bool/tristate selected to y, pinning it
+# -M- prompt - Tristate selected to m that also has m visibility,
+# pinning it to m
+# (foo) prompt - String/int/hex symbol with value "foo"
+# --> prompt - The selected symbol in a choice in y mode. This
+# syntax is unique to this example.
+#
+# When modules are disabled, the .type attribute of TRISTATE symbols and
+# choices automatically changes to BOOL. This trick is used by the C
+# implementation as well, and gives the expected behavior without having to do
+# anything extra here. The original type is available in .orig_type if needed.
+#
+# The Kconfiglib/examples/Kmenuconfig example uses named choices to be able to
+# refer to choices by name. Named choices are supported in the C tools too, but
+# I don't think I've ever seen them used in the wild.
+#
+# Sample session:
+#
+# $ python Kconfiglib/examples/menuconfig.py Kconfiglib/examples/Kmenuconfig
+#
+# ======== Example Kconfig configuration ========
+#
+# [*] Enable loadable module support (MODULES)
+# Bool and tristate symbols
+# [*] Bool symbol (BOOL)
+# [ ] Dependent bool symbol (BOOL_DEP)
+# < > Dependent tristate symbol (TRI_DEP)
+# [ ] First prompt (TWO_MENU_NODES)
+# < > Tristate symbol (TRI)
+# [ ] Second prompt (TWO_MENU_NODES)
+# *** These are selected by TRI_DEP ***
+# < > Tristate selected by TRI_DEP (SELECTED_BY_TRI_DEP)
+# < > Tristate implied by TRI_DEP (IMPLIED_BY_TRI_DEP)
+# String, int, and hex symbols
+# (foo) String symbol (STRING)
+# (747) Int symbol (INT)
+# (0xABC) Hex symbol (HEX)
+# Various choices
+# -*- Bool choice (BOOL_CHOICE)
+# --> Bool choice sym 1 (BOOL_CHOICE_SYM_1)
+# Bool choice sym 2 (BOOL_CHOICE_SYM_2)
+# {M} Tristate choice (TRI_CHOICE)
+# < > Tristate choice sym 1 (TRI_CHOICE_SYM_1)
+# < > Tristate choice sym 2 (TRI_CHOICE_SYM_2)
+# [ ] Optional bool choice (OPT_BOOL_CHOICE)
+#
+# Enter a symbol/choice name, "load_config", or "write_config" (or press CTRL+D to exit): BOOL
+# Value for BOOL (available: n, y): n
+#
+# ======== Example Kconfig configuration ========
+#
+# [*] Enable loadable module support (MODULES)
+# Bool and tristate symbols
+# [ ] Bool symbol (BOOL)
+# < > Tristate symbol (TRI)
+# [ ] Second prompt (TWO_MENU_NODES)
+# *** These are selected by TRI_DEP ***
+# < > Tristate selected by TRI_DEP (SELECTED_BY_TRI_DEP)
+# < > Tristate implied by TRI_DEP (IMPLIED_BY_TRI_DEP)
+# String, int, and hex symbols
+# (foo) String symbol (STRING)
+# (747) Int symbol (INT)
+# (0xABC) Hex symbol (HEX)
+# Various choices
+# -*- Bool choice (BOOL_CHOICE)
+# --> Bool choice sym 1 (BOOL_CHOICE_SYM_1)
+# Bool choice sym 2 (BOOL_CHOICE_SYM_2)
+# {M} Tristate choice (TRI_CHOICE)
+# < > Tristate choice sym 1 (TRI_CHOICE_SYM_1)
+# < > Tristate choice sym 2 (TRI_CHOICE_SYM_2)
+# [ ] Optional bool choice (OPT_BOOL_CHOICE)
+#
+# Enter a symbol/choice name, "load_config", or "write_config" (or press CTRL+D to exit): MODULES
+# Value for MODULES (available: n, y): n
+#
+# ======== Example Kconfig configuration ========
+#
+# [ ] Enable loadable module support (MODULES)
+# Bool and tristate symbols
+# [ ] Bool symbol (BOOL)
+# [ ] Tristate symbol (TRI)
+# [ ] Second prompt (TWO_MENU_NODES)
+# *** These are selected by TRI_DEP ***
+# [ ] Tristate selected by TRI_DEP (SELECTED_BY_TRI_DEP)
+# [ ] Tristate implied by TRI_DEP (IMPLIED_BY_TRI_DEP)
+# String, int, and hex symbols
+# (foo) String symbol (STRING)
+# (747) Int symbol (INT)
+# (0xABC) Hex symbol (HEX)
+# Various choices
+# -*- Bool choice (BOOL_CHOICE)
+# --> Bool choice sym 1 (BOOL_CHOICE_SYM_1)
+# Bool choice sym 2 (BOOL_CHOICE_SYM_2)
+# -*- Tristate choice (TRI_CHOICE)
+# --> Tristate choice sym 1 (TRI_CHOICE_SYM_1)
+# Tristate choice sym 2 (TRI_CHOICE_SYM_2)
+# [ ] Optional bool choice (OPT_BOOL_CHOICE)
+#
+# Enter a symbol/choice name, "load_config", or "write_config" (or press CTRL+D to exit): ^D
+
+from __future__ import print_function
+import readline
+import sys
+
+from kconfiglib import Kconfig, \
+ Symbol, MENU, COMMENT, \
+ BOOL, TRISTATE, STRING, INT, HEX, UNKNOWN, \
+ expr_value, \
+ TRI_TO_STR
+
+
+# Python 2/3 compatibility hack
+if sys.version_info[0] < 3:
+ input = raw_input
+
+
+def indent_print(s, indent):
+ print(indent*" " + s)
+
+
+def value_str(sc):
+ """
+ Returns the value part ("[*]", "<M>", "(foo)" etc.) of a menu entry.
+
+ sc: Symbol or Choice.
+ """
+ if sc.type in (STRING, INT, HEX):
+ return "({})".format(sc.str_value)
+
+ # BOOL or TRISTATE
+
+ # The choice mode is an upper bound on the visibility of choice symbols, so
+ # we can check the choice symbols' own visibility to see if the choice is
+ # in y mode
+ if isinstance(sc, Symbol) and sc.choice and sc.visibility == 2:
+ # For choices in y mode, print '-->' next to the selected symbol
+ return "-->" if sc.choice.selection is sc else " "
+
+ tri_val_str = (" ", "M", "*")[sc.tri_value]
+
+ if len(sc.assignable) == 1:
+ # Pinned to a single value
+ return "-{}-".format(tri_val_str)
+
+ if sc.type == BOOL:
+ return "[{}]".format(tri_val_str)
+
+ if sc.type == TRISTATE:
+ if sc.assignable == (1, 2):
+ # m and y available
+ return "{" + tri_val_str + "}" # Gets a bit confusing with .format()
+ return "<{}>".format(tri_val_str)
+
+
+def node_str(node):
+ """
+ Returns the complete menu entry text for a menu node, or "" for invisible
+ menu nodes. Invisible menu nodes are those that lack a prompt or that do
+ not have a satisfied prompt condition.
+
+ Example return value: "[*] Bool symbol (BOOL)"
+
+ The symbol name is printed in parentheses to the right of the prompt. This
+ is so that symbols can easily be referred to in the configuration
+ interface.
+ """
+ if not node.prompt:
+ return ""
+
+ # Even for menu nodes for symbols and choices, it's wrong to check
+ # Symbol.visibility / Choice.visibility here. The reason is that a symbol
+ # (and a choice, in theory) can be defined in multiple locations, giving it
+ # multiple menu nodes, which do not necessarily all have the same prompt
+ # visibility. Symbol.visibility / Choice.visibility is calculated as the OR
+ # of the visibility of all the prompts.
+ prompt, prompt_cond = node.prompt
+ if not expr_value(prompt_cond):
+ return ""
+
+ if node.item == MENU:
+ return " " + prompt
+
+ if node.item == COMMENT:
+ return " *** {} ***".format(prompt)
+
+ # Symbol or Choice
+
+ sc = node.item
+
+ if sc.type == UNKNOWN:
+ # Skip symbols defined without a type (these are obscure and generate
+ # a warning)
+ return ""
+
+ # {:3} sets the field width to three. Gives nice alignment for empty string
+ # values.
+ res = "{:3} {}".format(value_str(sc), prompt)
+
+ # Don't print the name for unnamed choices (the normal kind)
+ if sc.name is not None:
+ res += " ({})".format(sc.name)
+
+ return res
+
+
+def print_menuconfig_nodes(node, indent):
+ """
+ Prints a tree with all the menu entries rooted at 'node'. Child menu
+ entries are indented.
+ """
+ while node:
+ string = node_str(node)
+ if string:
+ indent_print(string, indent)
+
+ if node.list:
+ print_menuconfig_nodes(node.list, indent + 8)
+
+ node = node.next
+
+
+def print_menuconfig(kconf):
+ """
+ Prints all menu entries for the configuration.
+ """
+ # Print the expanded mainmenu text at the top. This is the same as
+ # kconf.top_node.prompt[0], but with variable references expanded.
+ print("\n======== {} ========\n".format(kconf.mainmenu_text))
+
+ print_menuconfig_nodes(kconf.top_node.list, 0)
+ print("")
+
+
+def get_value_from_user(sc):
+ """
+ Prompts the user for a value for the symbol or choice 'sc'. For
+ bool/tristate symbols and choices, provides a list of all the assignable
+ values.
+ """
+ if not sc.visibility:
+ print(sc.name + " is not currently visible")
+ return False
+
+ prompt = "Value for {}".format(sc.name)
+ if sc.type in (BOOL, TRISTATE):
+ prompt += " (available: {})" \
+ .format(", ".join(TRI_TO_STR[val] for val in sc.assignable))
+ prompt += ": "
+
+ val = input(prompt)
+
+ # Automatically add a "0x" prefix for hex symbols, like the menuconfig
+ # interface does. This isn't done when loading .config files, hence why
+ # set_value() doesn't do it automatically.
+ if sc.type == HEX and not val.startswith(("0x", "0X")):
+ val = "0x" + val
+
+ # Let Kconfiglib itself print a warning here if the value is invalid. We
+ # could also disable warnings temporarily with 'kconf.warn = False' and
+ # print our own warning.
+ return sc.set_value(val)
+
+
+if __name__ == "__main__":
+ if len(sys.argv) != 2:
+ sys.exit("usage: menuconfig.py <Kconfig file>")
+
+ # Load Kconfig configuration files
+ kconf = Kconfig(sys.argv[1])
+
+ # Print the initial configuration tree
+ print_menuconfig(kconf)
+
+ while True:
+ try:
+ cmd = input('Enter a symbol/choice name, "load_config", or '
+ '"write_config" (or press CTRL+D to exit): ').strip()
+ except EOFError:
+ print("")
+ break
+
+ if cmd == "load_config":
+ config_filename = input(".config file to load: ")
+ try:
+ # Returns a message telling which file got loaded
+ print(kconf.load_config(config_filename))
+ except EnvironmentError as e:
+ print(e, file=sys.stderr)
+
+ print_menuconfig(kconf)
+ continue
+
+ if cmd == "write_config":
+ config_filename = input("To this file: ")
+ try:
+ # Returns a message telling which file got saved
+ print(kconf.write_config(config_filename))
+ except EnvironmentError as e:
+ print(e, file=sys.stderr)
+
+ continue
+
+ # Assume 'cmd' is the name of a symbol or choice if it isn't one of the
+ # commands above, prompt the user for a value for it, and print the new
+ # configuration tree
+
+ if cmd in kconf.syms:
+ if get_value_from_user(kconf.syms[cmd]):
+ print_menuconfig(kconf)
+
+ continue
+
+ if cmd in kconf.named_choices:
+ if get_value_from_user(kconf.named_choices[cmd]):
+ print_menuconfig(kconf)
+
+ continue
+
+ print("No symbol/choice named '{}' in the configuration".format(cmd),
+ file=sys.stderr)
diff --git a/ext/Kconfiglib/examples/merge_config.py b/ext/Kconfiglib/examples/merge_config.py
new file mode 100755
index 0000000..777fe2c
--- /dev/null
+++ b/ext/Kconfiglib/examples/merge_config.py
@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+
+# This script functions similarly to scripts/kconfig/merge_config.sh from the
+# kernel tree, merging multiple configurations fragments to produce a complete
+# .config, with unspecified values filled in as for alldefconfig.
+#
+# The generated .config respects symbol dependencies, and a warning is printed
+# if any symbol gets a different value from the assigned value.
+#
+# For a real-world merging example based on this script, see
+# https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/kconfig/kconfig.py.
+#
+# Here's a demo:
+#
+# Kconfig contents:
+#
+# config FOO
+# bool "FOO"
+#
+# config BAR
+# bool "BAR"
+#
+# config BAZ
+# string "BAZ"
+#
+# config QAZ
+# bool "QAZ" if n
+#
+#
+# conf1 contents:
+#
+# CONFIG_FOO=y
+#
+#
+# conf2 contents:
+#
+# CONFIG_BAR=y
+#
+#
+# conf3 contents:
+#
+# # Assigned twice (would generate warning if 'warn_assign_override' was
+# # True)
+# # CONFIG_FOO is not set
+#
+# # Ops... this symbol doesn't exist
+# CONFIG_OPS=y
+#
+# CONFIG_BAZ="baz string"
+#
+#
+# conf4 contents:
+#
+# CONFIG_QAZ=y
+#
+#
+# Running:
+#
+# $ python(3) merge_config.py Kconfig merged conf1 conf2 conf3 conf4
+# Merged configuration 'conf1'
+# Merged configuration 'conf2'
+# conf3:5: warning: attempt to assign the value 'y' to the undefined symbol OPS
+# Merged configuration 'conf3'
+# Merged configuration 'conf4'
+# Configuration saved to 'merged'
+# warning: QAZ (defined at Kconfig:10) was assigned the value 'y' but got the value 'n' -- check dependencies
+# $ cat merged
+# Generated by Kconfiglib (https://github.com/ulfalizer/Kconfiglib)
+# # CONFIG_FOO is not set
+# CONFIG_BAR=y
+# CONFIG_BAZ="baz string"
+
+from __future__ import print_function
+import sys
+
+from kconfiglib import Kconfig, BOOL, TRISTATE, TRI_TO_STR
+
+
+if len(sys.argv) < 4:
+ sys.exit("usage: merge_config.py Kconfig merged_config config1 [config2 ...]")
+
+kconf = Kconfig(sys.argv[1], suppress_traceback=True)
+
+# Enable warnings for assignments to undefined symbols
+kconf.warn_assign_undef = True
+
+# (This script uses alldefconfig as the base. Other starting states could be
+# set up here as well. The approach in examples/allnoconfig_simpler.py could
+# provide an allnoconfig starting state for example.)
+
+# Disable warnings generated for multiple assignments to the same symbol within
+# a (set of) configuration files. Assigning a symbol multiple times might be
+# done intentionally when merging configuration files.
+kconf.warn_assign_override = False
+kconf.warn_assign_redun = False
+
+# Create a merged configuration by loading the fragments with replace=False.
+# load_config() and write_config() returns a message to print.
+for config in sys.argv[3:]:
+ print(kconf.load_config(config, replace=False))
+
+# Write the merged configuration
+print(kconf.write_config(sys.argv[2]))
+
+# Print warnings for symbols whose actual value doesn't match the assigned
+# value
+for sym in kconf.defined_syms:
+ # Was the symbol assigned to?
+ if sym.user_value is not None:
+ # Tristate values are represented as 0, 1, 2. Having them as
+ # "n", "m", "y" is more convenient here, so convert.
+ if sym.type in (BOOL, TRISTATE):
+ user_value = TRI_TO_STR[sym.user_value]
+ else:
+ user_value = sym.user_value
+
+ if user_value != sym.str_value:
+ print("warning: {} was assigned the value '{}' but got the "
+ "value '{}' -- check dependencies".format(
+ sym.name_and_loc, user_value, sym.str_value),
+ file=sys.stderr)
diff --git a/ext/Kconfiglib/examples/print_config_tree.py b/ext/Kconfiglib/examples/print_config_tree.py
new file mode 100644
index 0000000..dc81d9d
--- /dev/null
+++ b/ext/Kconfiglib/examples/print_config_tree.py
@@ -0,0 +1,199 @@
+# Prints menu entries as a tree with its value in the .config file. This can be
+# handy e.g. for diffing between different .config files or versions of Kconfig files.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=print_config_tree.py [SCRIPT_ARG=<.config>]
+#
+# If the variable WITH_HELP_DESC is modified to 'True', the help is added
+# to the symbols.
+#
+# Here's a notation guide. The notation matches the one used by menuconfig
+# (scripts/kconfig/mconf):
+#
+# [ ] prompt - Bool
+# < > prompt - Tristate
+# {M} prompt - Tristate selected to m. Can only be set to m or y.
+# -*- prompt - Bool/tristate selected to y, pinning it
+# -M- prompt - Tristate selected to m that also has m visibility,
+# pinning it to m
+# (foo) prompt - String/int/hex symbol with value "foo"
+# --> prompt - The selected symbol in a choice in y mode. This
+# syntax is unique to this example.
+#
+# When modules are disabled, the .type attribute of TRISTATE symbols and
+# choices automatically changes to BOOL. This trick is used by the C
+# implementation as well, and gives the expected behavior without having to do
+# anything extra here. The original type is available in .orig_type if needed.
+#
+# Example output:
+#
+# $ make scriptconfig SCRIPT=Kconfiglib/examples/print_config_tree.py [SCRIPT_ARG=<.config file>]
+#
+# ======== Linux/x86 4.9.82 Kernel Configuration ========
+#
+# [*] 64-bit kernel (64BIT)
+# General setup
+# () Cross-compiler tool prefix (CROSS_COMPILE)
+# [ ] Compile also drivers which will not load (COMPILE_TEST)
+# () Local version - append to kernel release (LOCALVERSION)
+# [*] Automatically append version information to the version string (LOCALVERSION_AUTO)
+# -*- Kernel compression mode
+# ...
+#
+# With the variable WITH_HELP_DESC modified to 'True':
+#
+# ======== Linux/x86 4.9.82 Kernel Configuration ========
+#
+# [*] 64-bit kernel - Say yes to build a 64-bit kernel - formerly known as x86_64 Say no to build a 32-bit kernel - formerly known as i386 (64BIT)
+# General setup
+# () Cross-compiler tool prefix - Same as running 'make CROSS_COMPILE=prefix-' but stored for default make runs in this kernel build directory. You don't need to set this unless you want the configured kernel build directory to select the cross-compiler automatically. (CROSS_COMPILE)
+# [ ] Compile also drivers which will not load - Some drivers can be compiled on a different platform than they are intended to be run on. Despite they cannot be loaded there (or even when they load they cannot be used due to missing HW support), developers still, opposing to distributors, might want to build such drivers to compile-test them. If you are a developer and want to build everything available, say Y here. If you are a user/distributor, say N here to exclude useless drivers to be distributed. (COMPILE_TEST)
+# ...
+
+import sys
+
+from kconfiglib import Kconfig, \
+ Symbol, MENU, COMMENT, \
+ BOOL, TRISTATE, STRING, INT, HEX, UNKNOWN, \
+ expr_value
+
+
+# Add help description to output
+WITH_HELP_DESC = False
+
+
+def indent_print(s, indent):
+ print(indent*" " + s)
+
+
+def value_str(sc):
+ """
+ Returns the value part ("[*]", "<M>", "(foo)" etc.) of a menu entry.
+
+ sc: Symbol or Choice.
+ """
+ if sc.type in (STRING, INT, HEX):
+ return "({})".format(sc.str_value)
+
+ # BOOL or TRISTATE
+
+ # The choice mode is an upper bound on the visibility of choice symbols, so
+ # we can check the choice symbols' own visibility to see if the choice is
+ # in y mode
+ if isinstance(sc, Symbol) and sc.choice and sc.visibility == 2:
+ # For choices in y mode, print '-->' next to the selected symbol
+ return "-->" if sc.choice.selection is sc else " "
+
+ tri_val_str = (" ", "M", "*")[sc.tri_value]
+
+ if len(sc.assignable) == 1:
+ # Pinned to a single value
+ return "-{}-".format(tri_val_str)
+
+ if sc.type == BOOL:
+ return "[{}]".format(tri_val_str)
+
+ if sc.type == TRISTATE:
+ if sc.assignable == (1, 2):
+ # m and y available
+ return "{" + tri_val_str + "}" # Gets a bit confusing with .format()
+ return "<{}>".format(tri_val_str)
+
+
+def node_str(node):
+ """
+ Returns the complete menu entry text for a menu node, or "" for invisible
+ menu nodes. Invisible menu nodes are those that lack a prompt or that do
+ not have a satisfied prompt condition.
+
+ Example return value: "[*] Bool symbol (BOOL)"
+
+ The symbol name is printed in parentheses to the right of the prompt.
+ """
+ if not node.prompt:
+ return ""
+
+ # Even for menu nodes for symbols and choices, it's wrong to check
+ # Symbol.visibility / Choice.visibility here. The reason is that a symbol
+ # (and a choice, in theory) can be defined in multiple locations, giving it
+ # multiple menu nodes, which do not necessarily all have the same prompt
+ # visibility. Symbol.visibility / Choice.visibility is calculated as the OR
+ # of the visibility of all the prompts.
+ prompt, prompt_cond = node.prompt
+ if not expr_value(prompt_cond):
+ return ""
+
+ if node.item == MENU:
+ return " " + prompt
+
+ if node.item == COMMENT:
+ return " *** {} ***".format(prompt)
+
+ # Symbol or Choice
+
+ sc = node.item
+
+ if sc.type == UNKNOWN:
+ # Skip symbols defined without a type (these are obscure and generate
+ # a warning)
+ return ""
+
+ # Add help text
+ if WITH_HELP_DESC:
+ prompt += ' - ' + str(node.help).replace('\n', ' ').replace('\r', '')
+
+ # {:3} sets the field width to three. Gives nice alignment for empty string
+ # values.
+ res = "{:3} {}".format(value_str(sc), prompt)
+
+ # Don't print the name for unnamed choices (the normal kind)
+ if sc.name is not None:
+ res += " ({})".format(sc.name)
+
+ return res
+
+
+def print_menuconfig_nodes(node, indent):
+ """
+ Prints a tree with all the menu entries rooted at 'node'. Child menu
+ entries are indented.
+ """
+ while node:
+ string = node_str(node)
+ if string:
+ indent_print(string, indent)
+
+ if node.list:
+ print_menuconfig_nodes(node.list, indent + 8)
+
+ node = node.next
+
+
+def print_menuconfig(kconf):
+ """
+ Prints all menu entries for the configuration.
+ """
+ # Print the expanded mainmenu text at the top. This is the same as
+ # kconf.top_node.prompt[0], but with variable references expanded.
+ print("\n======== {} ========\n".format(kconf.mainmenu_text))
+
+ print_menuconfig_nodes(kconf.top_node.list, 0)
+ print("")
+
+
+if __name__ == "__main__":
+
+ # Load Kconfig configuration files
+ kconf = Kconfig(sys.argv[1])
+
+ # Set default .config file or load it from argv
+ if len(sys.argv) == 2:
+ config_filename = '.config'
+ else:
+ config_filename = sys.argv[2]
+
+ kconf.load_config(config_filename)
+
+ # Print the configuration tree
+ print_menuconfig(kconf)
diff --git a/ext/Kconfiglib/examples/print_sym_info.py b/ext/Kconfiglib/examples/print_sym_info.py
new file mode 100644
index 0000000..ea6fc72
--- /dev/null
+++ b/ext/Kconfiglib/examples/print_sym_info.py
@@ -0,0 +1,54 @@
+# Loads a Kconfig and a .config and prints a symbol.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/print_sym_info.py SCRIPT_ARG=<name>
+#
+# Example output for SCRIPT_ARG=MODULES:
+#
+# menuconfig MODULES
+# bool
+# prompt "Enable loadable module support"
+# option modules
+# help
+# Kernel modules are small pieces of compiled code which can
+# be inserted in the running kernel, rather than being
+# permanently built into the kernel. You use the "modprobe"
+# tool to add (and sometimes remove) them. If you say Y here,
+# many parts of the kernel can be built as modules (by
+# answering M instead of Y where indicated): this is most
+# useful for infrequently used options which are not required
+# for booting. For more information, see the man pages for
+# modprobe, lsmod, modinfo, insmod and rmmod.
+#
+# If you say Y here, you will need to run "make
+# modules_install" to put the modules under /lib/modules/
+# where modprobe can find them (you may need to be root to do
+# this).
+#
+# If unsure, say Y.
+#
+# value = n
+# visibility = y
+# currently assignable values: n, y
+# defined at init/Kconfig:1674
+
+import sys
+
+from kconfiglib import Kconfig, TRI_TO_STR
+
+
+if len(sys.argv) < 3:
+ sys.exit('Pass symbol name (without "CONFIG_" prefix) with SCRIPT_ARG=<name>')
+
+kconf = Kconfig(sys.argv[1])
+sym = kconf.syms[sys.argv[2]]
+
+print(sym)
+print("value = " + sym.str_value)
+print("visibility = " + TRI_TO_STR[sym.visibility])
+print("currently assignable values: " +
+ ", ".join([TRI_TO_STR[v] for v in sym.assignable]))
+
+for node in sym.nodes:
+ print("defined at {}:{}".format(node.filename, node.linenr))
diff --git a/ext/Kconfiglib/examples/print_tree.py b/ext/Kconfiglib/examples/print_tree.py
new file mode 100644
index 0000000..49cb954
--- /dev/null
+++ b/ext/Kconfiglib/examples/print_tree.py
@@ -0,0 +1,75 @@
+# Prints the menu tree of the configuration. Dependencies between symbols can
+# sometimes implicitly alter the menu structure (see kconfig-language.txt), and
+# that's implemented too.
+#
+# Note: See the Kconfig.node_iter() function as well, which provides a simpler
+# interface for walking the menu tree.
+#
+# Usage:
+#
+# $ make [ARCH=<arch>] scriptconfig SCRIPT=Kconfiglib/examples/print_tree.py
+#
+# Example output:
+#
+# ...
+# config HAVE_KERNEL_LZO
+# config HAVE_KERNEL_LZ4
+# choice
+# config KERNEL_GZIP
+# config KERNEL_BZIP2
+# config KERNEL_LZMA
+# config KERNEL_XZ
+# config KERNEL_LZO
+# config KERNEL_LZ4
+# config DEFAULT_HOSTNAME
+# config SWAP
+# config SYSVIPC
+# config SYSVIPC_SYSCTL
+# config POSIX_MQUEUE
+# config POSIX_MQUEUE_SYSCTL
+# config CROSS_MEMORY_ATTACH
+# config FHANDLE
+# config USELIB
+# config AUDIT
+# config HAVE_ARCH_AUDITSYSCALL
+# config AUDITSYSCALL
+# config AUDIT_WATCH
+# config AUDIT_TREE
+# menu "IRQ subsystem"
+# config MAY_HAVE_SPARSE_IRQ
+# config GENERIC_IRQ_LEGACY
+# config GENERIC_IRQ_PROBE
+# ...
+
+import sys
+
+from kconfiglib import Kconfig, Symbol, Choice, MENU, COMMENT
+
+
+def indent_print(s, indent):
+ print(indent*" " + s)
+
+
+def print_items(node, indent):
+ while node:
+ if isinstance(node.item, Symbol):
+ indent_print("config " + node.item.name, indent)
+
+ elif isinstance(node.item, Choice):
+ indent_print("choice", indent)
+
+ elif node.item == MENU:
+ indent_print('menu "{}"'.format(node.prompt[0]), indent)
+
+ elif node.item == COMMENT:
+ indent_print('comment "{}"'.format(node.prompt[0]), indent)
+
+
+ if node.list:
+ print_items(node.list, indent + 2)
+
+ node = node.next
+
+
+kconf = Kconfig(sys.argv[1])
+print_items(kconf.top_node, 0)
diff --git a/ext/Kconfiglib/gem5notes.txt b/ext/Kconfiglib/gem5notes.txt
new file mode 100644
index 0000000..b3ad870
--- /dev/null
+++ b/ext/Kconfiglib/gem5notes.txt
@@ -0,0 +1,15 @@
+Two symlinks have been deleted from this kconfig distribution because they
+caused problems with git:
+
+tests/symlink => sub/sub
+examples/kconfiglib.py -> ../kconfiglib.py
+
+To run kconfig's tests, you will likely need to recreate tests/symlink. To
+run the examples, you will need to make sure kconfiglib.py is in the import
+path, either by recreating examples/kconfiglib.py, or by adjusting the module
+search path at the python level.
+
+Also, to avoid adding unnecessary python files to the root of the module search
+path, this change also moves kconfiglib.py into a directory called "import"
+where it lives by itself. The examples/kconfiglib.py symlink would need to
+point to the library in this new location, if recreated.
diff --git a/ext/Kconfiglib/genconfig.py b/ext/Kconfiglib/genconfig.py
new file mode 100755
index 0000000..62f065b
--- /dev/null
+++ b/ext/Kconfiglib/genconfig.py
@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2018-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Generates a header file with #defines from the configuration, matching the
+format of include/generated/autoconf.h in the Linux kernel.
+
+Optionally, also writes the configuration output as a .config file. See
+--config-out.
+
+The --sync-deps, --file-list, and --env-list options generate information that
+can be used to avoid needless rebuilds/reconfigurations.
+
+Before writing a header or configuration file, Kconfiglib compares the old
+contents of the file against the new contents. If there's no change, the write
+is skipped. This avoids updating file metadata like the modification time, and
+might save work depending on your build setup.
+
+By default, the configuration is generated from '.config'. A different
+configuration file can be passed in the KCONFIG_CONFIG environment variable.
+
+A custom header string can be inserted at the beginning of generated
+configuration and header files by setting the KCONFIG_CONFIG_HEADER and
+KCONFIG_AUTOHEADER_HEADER environment variables, respectively (this also works
+for other scripts). The string is not automatically made a comment (this is by
+design, to allow anything to be added), and no trailing newline is added, so
+add '/* */', '#', and newlines as appropriate.
+
+See https://www.gnu.org/software/make/manual/make.html#Multi_002dLine for a
+handy way to define multi-line variables in makefiles, for use with custom
+headers. Remember to export the variable to the environment.
+"""
+import argparse
+import os
+import sys
+
+import kconfiglib
+
+
+DEFAULT_SYNC_DEPS_PATH = "deps/"
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ description=__doc__)
+
+ parser.add_argument(
+ "--header-path",
+ metavar="HEADER_FILE",
+ help="""
+Path to write the generated header file to. If not specified, the path in the
+environment variable KCONFIG_AUTOHEADER is used if it is set, and 'config.h'
+otherwise.
+""")
+
+ parser.add_argument(
+ "--config-out",
+ metavar="CONFIG_FILE",
+ help="""
+Write the configuration to CONFIG_FILE. This is useful if you include .config
+files in Makefiles, as the generated configuration file will be a full .config
+file even if .config is outdated. The generated configuration matches what
+olddefconfig would produce. If you use sync-deps, you can include
+deps/auto.conf instead. --config-out is meant for cases where incremental build
+information isn't needed.
+""")
+
+ parser.add_argument(
+ "--sync-deps",
+ metavar="OUTPUT_DIR",
+ nargs="?",
+ const=DEFAULT_SYNC_DEPS_PATH,
+ help="""
+Enable generation of symbol dependency information for incremental builds,
+optionally specifying the output directory (default: {}). See the docstring of
+Kconfig.sync_deps() in Kconfiglib for more information.
+""".format(DEFAULT_SYNC_DEPS_PATH))
+
+ parser.add_argument(
+ "--file-list",
+ metavar="OUTPUT_FILE",
+ help="""
+Write a list of all Kconfig files to OUTPUT_FILE, with one file per line. The
+paths are relative to $srctree (or to the current directory if $srctree is
+unset). Files appear in the order they're 'source'd.
+""")
+
+ parser.add_argument(
+ "--env-list",
+ metavar="OUTPUT_FILE",
+ help="""
+Write a list of all environment variables referenced in Kconfig files to
+OUTPUT_FILE, with one variable per line. Each line has the format NAME=VALUE.
+Only environment variables referenced with the preprocessor $(VAR) syntax are
+included, and not variables referenced with the older $VAR syntax (which is
+only supported for backwards compatibility).
+""")
+
+ parser.add_argument(
+ "kconfig",
+ metavar="KCONFIG",
+ nargs="?",
+ default="Kconfig",
+ help="Top-level Kconfig file (default: Kconfig)")
+
+ args = parser.parse_args()
+
+
+ kconf = kconfiglib.Kconfig(args.kconfig, suppress_traceback=True)
+ kconf.load_config()
+
+ if args.header_path is None:
+ if "KCONFIG_AUTOHEADER" in os.environ:
+ kconf.write_autoconf()
+ else:
+ # Kconfiglib defaults to include/generated/autoconf.h to be
+ # compatible with the C tools. 'config.h' is used here instead for
+ # backwards compatibility. It's probably a saner default for tools
+ # as well.
+ kconf.write_autoconf("config.h")
+ else:
+ kconf.write_autoconf(args.header_path)
+
+ if args.config_out is not None:
+ kconf.write_config(args.config_out, save_old=False)
+
+ if args.sync_deps is not None:
+ kconf.sync_deps(args.sync_deps)
+
+ if args.file_list is not None:
+ with _open_write(args.file_list) as f:
+ for path in kconf.kconfig_filenames:
+ f.write(path + "\n")
+
+ if args.env_list is not None:
+ with _open_write(args.env_list) as f:
+ for env_var in kconf.env_vars:
+ f.write("{}={}\n".format(env_var, os.environ[env_var]))
+
+
+def _open_write(path):
+ # Python 2/3 compatibility. io.open() is available on both, but makes
+ # write() expect 'unicode' strings on Python 2.
+
+ if sys.version_info[0] < 3:
+ return open(path, "w")
+ return open(path, "w", encoding="utf-8")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ext/Kconfiglib/guiconfig.py b/ext/Kconfiglib/guiconfig.py
new file mode 100755
index 0000000..7804fdc
--- /dev/null
+++ b/ext/Kconfiglib/guiconfig.py
@@ -0,0 +1,2324 @@
+#!/usr/bin/env python3
+
+# Copyright (c) 2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Overview
+========
+
+A Tkinter-based menuconfig implementation, based around a treeview control and
+a help display. The interface should feel familiar to people used to qconf
+('make xconfig'). Compatible with both Python 2 and Python 3.
+
+The display can be toggled between showing the full tree and showing just a
+single menu (like menuconfig.py). Only single-menu mode distinguishes between
+symbols defined with 'config' and symbols defined with 'menuconfig'.
+
+A show-all mode is available that shows invisible items in red.
+
+Supports both mouse and keyboard controls. The following keyboard shortcuts are
+available:
+
+ Ctrl-S : Save configuration
+ Ctrl-O : Open configuration
+ Ctrl-A : Toggle show-all mode
+ Ctrl-N : Toggle show-name mode
+ Ctrl-M : Toggle single-menu mode
+ Ctrl-F, /: Open jump-to dialog
+ ESC : Close
+
+Running
+=======
+
+guiconfig.py can be run either as a standalone executable or by calling the
+menuconfig() function with an existing Kconfig instance. The second option is a
+bit inflexible in that it will still load and save .config, etc.
+
+When run in standalone mode, the top-level Kconfig file to load can be passed
+as a command-line argument. With no argument, it defaults to "Kconfig".
+
+The KCONFIG_CONFIG environment variable specifies the .config file to load (if
+it exists) and save. If KCONFIG_CONFIG is unset, ".config" is used.
+
+When overwriting a configuration file, the old version is saved to
+<filename>.old (e.g. .config.old).
+
+$srctree is supported through Kconfiglib.
+"""
+
+# Note: There's some code duplication with menuconfig.py below, especially for
+# the help text. Maybe some of it could be moved into kconfiglib.py or a shared
+# helper script, but OTOH it's pretty nice to have things standalone and
+# customizable.
+
+import errno
+import os
+import sys
+
+_PY2 = sys.version_info[0] < 3
+
+if _PY2:
+ # Python 2
+ from Tkinter import *
+ import ttk
+ import tkFont as font
+ import tkFileDialog as filedialog
+ import tkMessageBox as messagebox
+else:
+ # Python 3
+ from tkinter import *
+ import tkinter.ttk as ttk
+ import tkinter.font as font
+ from tkinter import filedialog, messagebox
+
+from kconfiglib import Symbol, Choice, MENU, COMMENT, MenuNode, \
+ BOOL, TRISTATE, STRING, INT, HEX, \
+ AND, OR, \
+ expr_str, expr_value, split_expr, \
+ standard_sc_expr_str, \
+ TRI_TO_STR, TYPE_TO_STR, \
+ standard_kconfig, standard_config_filename
+
+
+# If True, use GIF image data embedded in this file instead of separate GIF
+# files. See _load_images().
+_USE_EMBEDDED_IMAGES = True
+
+
+# Help text for the jump-to dialog
+_JUMP_TO_HELP = """\
+Type one or more strings/regexes and press Enter to list items that match all
+of them. Python's regex flavor is used (see the 're' module). Double-clicking
+an item will jump to it. Item values can be toggled directly within the dialog.\
+"""
+
+
+def _main():
+ menuconfig(standard_kconfig(__doc__))
+
+
+# Global variables used below:
+#
+# _root:
+# The Toplevel instance for the main window
+#
+# _tree:
+# The Treeview in the main window
+#
+# _jump_to_tree:
+# The Treeview in the jump-to dialog. None if the jump-to dialog isn't
+# open. Doubles as a flag.
+#
+# _jump_to_matches:
+# List of Nodes shown in the jump-to dialog
+#
+# _menupath:
+# The Label that shows the menu path of the selected item
+#
+# _backbutton:
+# The button shown in single-menu mode for jumping to the parent menu
+#
+# _status_label:
+# Label with status text shown at the bottom of the main window
+# ("Modified", "Saved to ...", etc.)
+#
+# _id_to_node:
+# We can't use Node objects directly as Treeview item IDs, so we use their
+# id()s instead. This dictionary maps Node id()s back to Nodes. (The keys
+# are actually str(id(node)), just to simplify lookups.)
+#
+# _cur_menu:
+# The current menu. Ignored outside single-menu mode.
+#
+# _show_all_var/_show_name_var/_single_menu_var:
+# Tkinter Variable instances bound to the corresponding checkboxes
+#
+# _show_all/_single_menu:
+# Plain Python bools that track _show_all_var and _single_menu_var, to
+# speed up and simplify things a bit
+#
+# _conf_filename:
+# File to save the configuration to
+#
+# _minconf_filename:
+# File to save minimal configurations to
+#
+# _conf_changed:
+# True if the configuration has been changed. If False, we don't bother
+# showing the save-and-quit dialog.
+#
+# We reset this to False whenever the configuration is saved.
+#
+# _*_img:
+# PhotoImage instances for images
+
+
+def menuconfig(kconf):
+ """
+ Launches the configuration interface, returning after the user exits.
+
+ kconf:
+ Kconfig instance to be configured
+ """
+ global _kconf
+ global _conf_filename
+ global _minconf_filename
+ global _jump_to_tree
+ global _cur_menu
+
+ _kconf = kconf
+
+ _jump_to_tree = None
+
+ _create_id_to_node()
+
+ _create_ui()
+
+ # Filename to save configuration to
+ _conf_filename = standard_config_filename()
+
+ # Load existing configuration and check if it's outdated
+ _set_conf_changed(_load_config())
+
+ # Filename to save minimal configuration to
+ _minconf_filename = "defconfig"
+
+ # Current menu in single-menu mode
+ _cur_menu = _kconf.top_node
+
+ # Any visible items in the top menu?
+ if not _shown_menu_nodes(kconf.top_node):
+ # Nothing visible. Start in show-all mode and try again.
+ _show_all_var.set(True)
+ if not _shown_menu_nodes(kconf.top_node):
+ # Give up and show an error. It's nice to be able to assume that
+ # the tree is non-empty in the rest of the code.
+ _root.wait_visibility()
+ messagebox.showerror(
+ "Error",
+ "Empty configuration -- nothing to configure.\n\n"
+ "Check that environment variables are set properly.")
+ _root.destroy()
+ return
+
+ # Build the initial tree
+ _update_tree()
+
+ # Select the first item and focus the Treeview, so that keyboard controls
+ # work immediately
+ _select(_tree, _tree.get_children()[0])
+ _tree.focus_set()
+
+ # Make geometry information available for centering the window. This
+ # indirectly creates the window, so hide it so that it's never shown at the
+ # old location.
+ _root.withdraw()
+ _root.update_idletasks()
+
+ # Center the window
+ _root.geometry("+{}+{}".format(
+ (_root.winfo_screenwidth() - _root.winfo_reqwidth())//2,
+ (_root.winfo_screenheight() - _root.winfo_reqheight())//2))
+
+ # Show it
+ _root.deiconify()
+
+ # Prevent the window from being automatically resized. Otherwise, it
+ # changes size when scrollbars appear/disappear before the user has
+ # manually resized it.
+ _root.geometry(_root.geometry())
+
+ _root.mainloop()
+
+
+def _load_config():
+ # Loads any existing .config file. See the Kconfig.load_config() docstring.
+ #
+ # Returns True if .config is missing or outdated. We always prompt for
+ # saving the configuration in that case.
+
+ print(_kconf.load_config())
+ if not os.path.exists(_conf_filename):
+ # No .config
+ return True
+
+ return _needs_save()
+
+
+def _needs_save():
+ # Returns True if a just-loaded .config file is outdated (would get
+ # modified when saving)
+
+ if _kconf.missing_syms:
+ # Assignments to undefined symbols in the .config
+ return True
+
+ for sym in _kconf.unique_defined_syms:
+ if sym.user_value is None:
+ if sym.config_string:
+ # Unwritten symbol
+ return True
+ elif sym.orig_type in (BOOL, TRISTATE):
+ if sym.tri_value != sym.user_value:
+ # Written bool/tristate symbol, new value
+ return True
+ elif sym.str_value != sym.user_value:
+ # Written string/int/hex symbol, new value
+ return True
+
+ # No need to prompt for save
+ return False
+
+
+def _create_id_to_node():
+ global _id_to_node
+
+ _id_to_node = {str(id(node)): node for node in _kconf.node_iter()}
+
+
+def _create_ui():
+ # Creates the main window UI
+
+ global _root
+ global _tree
+
+ # Create the root window. This initializes Tkinter and makes e.g.
+ # PhotoImage available, so do it early.
+ _root = Tk()
+
+ _load_images()
+ _init_misc_ui()
+ _fix_treeview_issues()
+
+ _create_top_widgets()
+ # Create the pane with the Kconfig tree and description text
+ panedwindow, _tree = _create_kconfig_tree_and_desc(_root)
+ panedwindow.grid(column=0, row=1, sticky="nsew")
+ _create_status_bar()
+
+ _root.columnconfigure(0, weight=1)
+ # Only the pane with the Kconfig tree and description grows vertically
+ _root.rowconfigure(1, weight=1)
+
+ # Start with show-name disabled
+ _do_showname()
+
+ _tree.bind("<Left>", _tree_left_key)
+ _tree.bind("<Right>", _tree_right_key)
+ # Note: Binding this for the jump-to tree as well would cause issues due to
+ # the Tk bug mentioned in _tree_open()
+ _tree.bind("<<TreeviewOpen>>", _tree_open)
+ # add=True to avoid overriding the description text update
+ _tree.bind("<<TreeviewSelect>>", _update_menu_path, add=True)
+
+ _root.bind("<Control-s>", _save)
+ _root.bind("<Control-o>", _open)
+ _root.bind("<Control-a>", _toggle_showall)
+ _root.bind("<Control-n>", _toggle_showname)
+ _root.bind("<Control-m>", _toggle_tree_mode)
+ _root.bind("<Control-f>", _jump_to_dialog)
+ _root.bind("/", _jump_to_dialog)
+ _root.bind("<Escape>", _on_quit)
+
+
+def _load_images():
+ # Loads GIF images, creating the global _*_img PhotoImage variables.
+ # Base64-encoded images embedded in this script are used if
+ # _USE_EMBEDDED_IMAGES is True, and separate image files in the same
+ # directory as the script otherwise.
+ #
+ # Using a global variable indirectly prevents the image from being
+ # garbage-collected. Passing an image to a Tkinter function isn't enough to
+ # keep it alive.
+
+ def load_image(name, data):
+ var_name = "_{}_img".format(name)
+
+ if _USE_EMBEDDED_IMAGES:
+ globals()[var_name] = PhotoImage(data=data, format="gif")
+ else:
+ globals()[var_name] = PhotoImage(
+ file=os.path.join(os.path.dirname(__file__), name + ".gif"),
+ format="gif")
+
+ # Note: Base64 data can be put on the clipboard with
+ # $ base64 -w0 foo.gif | xclip
+
+ load_image("icon", "R0lGODlhMAAwAPEDAAAAAADQAO7u7v///yH5BAUKAAMALAAAAAAwADAAAAL/nI+gy+2Pokyv2jazuZxryQjiSJZmyXxHeLbumH6sEATvW8OLNtf5bfLZRLFITzgEipDJ4mYxYv6A0ubuqYhWk66tVTE4enHer7jcKvt0LLUw6P45lvEprT6c0+v7OBuqhYdHohcoqIbSAHc4ljhDwrh1UlgSydRCWWlp5wiYZvmSuSh4IzrqV6p4cwhkCsmY+nhK6uJ6t1mrOhuJqfu6+WYiCiwl7HtLjNSZZZis/MeM7NY3TaRKS40ooDeoiVqIultsrav92bi9c3a5KkkOsOJZpSS99m4k/0zPng4Gks9JSbB+8DIcoQfnjwpZCHv5W+ip4aQrKrB0uOikYhiMCBw1/uPoQUMBADs=")
+ load_image("n_bool", "R0lGODdhEAAQAPAAAAgICP///ywAAAAAEAAQAAACIISPacHtvp5kcb5qG85hZ2+BkyiRF8BBaEqtrKkqslEAADs=")
+ load_image("y_bool", "R0lGODdhEAAQAPEAAAgICADQAP///wAAACwAAAAAEAAQAAACMoSPacLtvlh4YrIYsst2cV19AvaVF9CUXBNJJoum7ymrsKuCnhiupIWjSSjAFuWhSCIKADs=")
+ load_image("n_tri", "R0lGODlhEAAQAPD/AAEBAf///yH5BAUKAAIALAAAAAAQABAAAAInlI+pBrAKQnCPSUlXvFhznlkfeGwjKZhnJ65h6nrfi6h0st2QXikFADs=")
+ load_image("m_tri", "R0lGODlhEAAQAPEDAAEBAeQMuv///wAAACH5BAUKAAMALAAAAAAQABAAAAI5nI+pBrAWAhPCjYhiAJQCnWmdoElHGVBoiK5M21ofXFpXRIrgiecqxkuNciZIhNOZFRNI24PhfEoLADs=")
+ load_image("y_tri", "R0lGODlhEAAQAPEDAAICAgDQAP///wAAACH5BAUKAAMALAAAAAAQABAAAAI0nI+pBrAYBhDCRRUypfmergmgZ4xjMpmaw2zmxk7cCB+pWiVqp4MzDwn9FhGZ5WFjIZeGAgA7")
+ load_image("m_my", "R0lGODlhEAAQAPEDAAAAAOQMuv///wAAACH5BAUKAAMALAAAAAAQABAAAAI5nIGpxiAPI2ghxFinq/ZygQhc94zgZopmOLYf67anGr+oZdp02emfV5n9MEHN5QhqICETxkABbQ4KADs=")
+ load_image("y_my", "R0lGODlhEAAQAPH/AAAAAADQAAPRA////yH5BAUKAAQALAAAAAAQABAAAAM+SArcrhCMSSuIM9Q8rxxBWIXawIBkmWonupLd565Um9G1PIs59fKmzw8WnAlusBYR2SEIN6DmAmqBLBxYSAIAOw==")
+ load_image("n_locked", "R0lGODlhEAAQAPABAAAAAP///yH5BAUKAAEALAAAAAAQABAAAAIgjB8AyKwN04pu0vMutpqqz4Hih4ydlnUpyl2r23pxUAAAOw==")
+ load_image("m_locked", "R0lGODlhEAAQAPD/AAAAAOQMuiH5BAUKAAIALAAAAAAQABAAAAIylC8AyKwN04ohnGcqqlZmfXDWI26iInZoyiore05walolV39ftxsYHgL9QBBMBGFEFAAAOw==")
+ load_image("y_locked", "R0lGODlhEAAQAPD/AAAAAADQACH5BAUKAAIALAAAAAAQABAAAAIylC8AyKzNgnlCtoDTwvZwrHydIYpQmR3KWq4uK74IOnp0HQPmnD3cOVlUIAgKsShkFAAAOw==")
+ load_image("not_selected", "R0lGODlhEAAQAPD/AAAAAP///yH5BAUKAAIALAAAAAAQABAAAAIrlA2px6IBw2IpWglOvTYhzmUbGD3kNZ5QqrKn2YrqigCxZoMelU6No9gdCgA7")
+ load_image("selected", "R0lGODlhEAAQAPD/AAAAAP///yH5BAUKAAIALAAAAAAQABAAAAIzlA2px6IBw2IpWglOvTah/kTZhimASJomiqonlLov1qptHTsgKSEzh9H8QI0QzNPwmRoFADs=")
+ load_image("edit", "R0lGODlhEAAQAPIFAAAAAKOLAMuuEPvXCvrxvgAAAAAAAAAAACH5BAUKAAUALAAAAAAQABAAAANCWLqw/gqMBp8cszJxcwVC2FEOEIAi5kVBi3IqWZhuCGMyfdpj2e4pnK+WAshmvxeAcETWlsxPkkBtsqBMa8TIBSQAADs=")
+
+
+def _fix_treeview_issues():
+ # Fixes some Treeview issues
+
+ global _treeview_rowheight
+
+ style = ttk.Style()
+
+ # The treeview rowheight isn't adjusted automatically on high-DPI displays,
+ # so do it ourselves. The font will probably always be TkDefaultFont, but
+ # play it safe and look it up.
+
+ _treeview_rowheight = font.Font(font=style.lookup("Treeview", "font")) \
+ .metrics("linespace") + 2
+
+ style.configure("Treeview", rowheight=_treeview_rowheight)
+
+ # Work around regression in https://core.tcl.tk/tk/tktview?name=509cafafae,
+ # which breaks tag background colors
+
+ for option in "foreground", "background":
+ # Filter out any styles starting with ("!disabled", "!selected", ...).
+ # style.map() returns an empty list for missing options, so this should
+ # be future-safe.
+ style.map(
+ "Treeview",
+ **{option: [elm for elm in style.map("Treeview", query_opt=option)
+ if elm[:2] != ("!disabled", "!selected")]})
+
+
+def _init_misc_ui():
+ # Does misc. UI initialization, like setting the title, icon, and theme
+
+ _root.title(_kconf.mainmenu_text)
+ # iconphoto() isn't available in Python 2's Tkinter
+ _root.tk.call("wm", "iconphoto", _root._w, "-default", _icon_img)
+ # Reducing the width of the window to 1 pixel makes it move around, at
+ # least on GNOME. Prevent weird stuff like that.
+ _root.minsize(128, 128)
+ _root.protocol("WM_DELETE_WINDOW", _on_quit)
+
+ # Use the 'clam' theme on *nix if it's available. It looks nicer than the
+ # 'default' theme.
+ if _root.tk.call("tk", "windowingsystem") == "x11":
+ style = ttk.Style()
+ if "clam" in style.theme_names():
+ style.theme_use("clam")
+
+
+def _create_top_widgets():
+ # Creates the controls above the Kconfig tree in the main window
+
+ global _show_all_var
+ global _show_name_var
+ global _single_menu_var
+ global _menupath
+ global _backbutton
+
+ topframe = ttk.Frame(_root)
+ topframe.grid(column=0, row=0, sticky="ew")
+
+ ttk.Button(topframe, text="Save", command=_save) \
+ .grid(column=0, row=0, sticky="ew", padx=".05c", pady=".05c")
+
+ ttk.Button(topframe, text="Save as...", command=_save_as) \
+ .grid(column=1, row=0, sticky="ew")
+
+ ttk.Button(topframe, text="Save minimal (advanced)...",
+ command=_save_minimal) \
+ .grid(column=2, row=0, sticky="ew", padx=".05c")
+
+ ttk.Button(topframe, text="Open...", command=_open) \
+ .grid(column=3, row=0)
+
+ ttk.Button(topframe, text="Jump to...", command=_jump_to_dialog) \
+ .grid(column=4, row=0, padx=".05c")
+
+ _show_name_var = BooleanVar()
+ ttk.Checkbutton(topframe, text="Show name", command=_do_showname,
+ variable=_show_name_var) \
+ .grid(column=0, row=1, sticky="nsew", padx=".05c", pady="0 .05c",
+ ipady=".2c")
+
+ _show_all_var = BooleanVar()
+ ttk.Checkbutton(topframe, text="Show all", command=_do_showall,
+ variable=_show_all_var) \
+ .grid(column=1, row=1, sticky="nsew", pady="0 .05c")
+
+ # Allow the show-all and single-menu status to be queried via plain global
+ # Python variables, which is faster and simpler
+
+ def show_all_updated(*_):
+ global _show_all
+ _show_all = _show_all_var.get()
+
+ _trace_write(_show_all_var, show_all_updated)
+ _show_all_var.set(False)
+
+ _single_menu_var = BooleanVar()
+ ttk.Checkbutton(topframe, text="Single-menu mode", command=_do_tree_mode,
+ variable=_single_menu_var) \
+ .grid(column=2, row=1, sticky="nsew", padx=".05c", pady="0 .05c")
+
+ _backbutton = ttk.Button(topframe, text="<--", command=_leave_menu,
+ state="disabled")
+ _backbutton.grid(column=0, row=4, sticky="nsew", padx=".05c", pady="0 .05c")
+
+ def tree_mode_updated(*_):
+ global _single_menu
+ _single_menu = _single_menu_var.get()
+
+ if _single_menu:
+ _backbutton.grid()
+ else:
+ _backbutton.grid_remove()
+
+ _trace_write(_single_menu_var, tree_mode_updated)
+ _single_menu_var.set(False)
+
+ # Column to the right of the buttons that the menu path extends into, so
+ # that it can grow wider than the buttons
+ topframe.columnconfigure(5, weight=1)
+
+ _menupath = ttk.Label(topframe)
+ _menupath.grid(column=0, row=3, columnspan=6, sticky="w", padx="0.05c",
+ pady="0 .05c")
+
+
+def _create_kconfig_tree_and_desc(parent):
+ # Creates a Panedwindow with a Treeview that shows Kconfig nodes and a Text
+ # that shows a description of the selected node. Returns a tuple with the
+ # Panedwindow and the Treeview. This code is shared between the main window
+ # and the jump-to dialog.
+
+ panedwindow = ttk.Panedwindow(parent, orient=VERTICAL)
+
+ tree_frame, tree = _create_kconfig_tree(panedwindow)
+ desc_frame, desc = _create_kconfig_desc(panedwindow)
+
+ panedwindow.add(tree_frame, weight=1)
+ panedwindow.add(desc_frame)
+
+ def tree_select(_):
+ # The Text widget does not allow editing the text in its disabled
+ # state. We need to temporarily enable it.
+ desc["state"] = "normal"
+
+ sel = tree.selection()
+ if not sel:
+ desc.delete("1.0", "end")
+ desc["state"] = "disabled"
+ return
+
+ # Text.replace() is not available in Python 2's Tkinter
+ desc.delete("1.0", "end")
+ desc.insert("end", _info_str(_id_to_node[sel[0]]))
+
+ desc["state"] = "disabled"
+
+ tree.bind("<<TreeviewSelect>>", tree_select)
+ tree.bind("<1>", _tree_click)
+ tree.bind("<Double-1>", _tree_double_click)
+ tree.bind("<Return>", _tree_enter)
+ tree.bind("<KP_Enter>", _tree_enter)
+ tree.bind("<space>", _tree_toggle)
+ tree.bind("n", _tree_set_val(0))
+ tree.bind("m", _tree_set_val(1))
+ tree.bind("y", _tree_set_val(2))
+
+ return panedwindow, tree
+
+
+def _create_kconfig_tree(parent):
+ # Creates a Treeview for showing Kconfig nodes
+
+ frame = ttk.Frame(parent)
+
+ tree = ttk.Treeview(frame, selectmode="browse", height=20,
+ columns=("name",))
+ tree.heading("#0", text="Option", anchor="w")
+ tree.heading("name", text="Name", anchor="w")
+
+ tree.tag_configure("n-bool", image=_n_bool_img)
+ tree.tag_configure("y-bool", image=_y_bool_img)
+ tree.tag_configure("m-tri", image=_m_tri_img)
+ tree.tag_configure("n-tri", image=_n_tri_img)
+ tree.tag_configure("m-tri", image=_m_tri_img)
+ tree.tag_configure("y-tri", image=_y_tri_img)
+ tree.tag_configure("m-my", image=_m_my_img)
+ tree.tag_configure("y-my", image=_y_my_img)
+ tree.tag_configure("n-locked", image=_n_locked_img)
+ tree.tag_configure("m-locked", image=_m_locked_img)
+ tree.tag_configure("y-locked", image=_y_locked_img)
+ tree.tag_configure("not-selected", image=_not_selected_img)
+ tree.tag_configure("selected", image=_selected_img)
+ tree.tag_configure("edit", image=_edit_img)
+ tree.tag_configure("invisible", foreground="red")
+
+ tree.grid(column=0, row=0, sticky="nsew")
+
+ _add_vscrollbar(frame, tree)
+
+ frame.columnconfigure(0, weight=1)
+ frame.rowconfigure(0, weight=1)
+
+ # Create items for all menu nodes. These can be detached/moved later.
+ # Micro-optimize this a bit.
+ insert = tree.insert
+ id_ = id
+ Symbol_ = Symbol
+ for node in _kconf.node_iter():
+ item = node.item
+ insert("", "end", iid=id_(node),
+ values=item.name if item.__class__ is Symbol_ else "")
+
+ return frame, tree
+
+
+def _create_kconfig_desc(parent):
+ # Creates a Text for showing the description of the selected Kconfig node
+
+ frame = ttk.Frame(parent)
+
+ desc = Text(frame, height=12, wrap="none", borderwidth=0,
+ state="disabled")
+ desc.grid(column=0, row=0, sticky="nsew")
+
+ # Work around not being to Ctrl-C/V text from a disabled Text widget, with a
+ # tip found in https://stackoverflow.com/questions/3842155/is-there-a-way-to-make-the-tkinter-text-widget-read-only
+ desc.bind("<1>", lambda _: desc.focus_set())
+
+ _add_vscrollbar(frame, desc)
+
+ frame.columnconfigure(0, weight=1)
+ frame.rowconfigure(0, weight=1)
+
+ return frame, desc
+
+
+def _add_vscrollbar(parent, widget):
+ # Adds a vertical scrollbar to 'widget' that's only shown as needed
+
+ vscrollbar = ttk.Scrollbar(parent, orient="vertical",
+ command=widget.yview)
+ vscrollbar.grid(column=1, row=0, sticky="ns")
+
+ def yscrollcommand(first, last):
+ # Only show the scrollbar when needed. 'first' and 'last' are
+ # strings.
+ if float(first) <= 0.0 and float(last) >= 1.0:
+ vscrollbar.grid_remove()
+ else:
+ vscrollbar.grid()
+
+ vscrollbar.set(first, last)
+
+ widget["yscrollcommand"] = yscrollcommand
+
+
+def _create_status_bar():
+ # Creates the status bar at the bottom of the main window
+
+ global _status_label
+
+ _status_label = ttk.Label(_root, anchor="e", padding="0 0 0.4c 0")
+ _status_label.grid(column=0, row=3, sticky="ew")
+
+
+def _set_status(s):
+ # Sets the text in the status bar to 's'
+
+ _status_label["text"] = s
+
+
+def _set_conf_changed(changed):
+ # Updates the status re. whether there are unsaved changes
+
+ global _conf_changed
+
+ _conf_changed = changed
+ if changed:
+ _set_status("Modified")
+
+
+def _update_tree():
+ # Updates the Kconfig tree in the main window by first detaching all nodes
+ # and then updating and reattaching them. The tree structure might have
+ # changed.
+
+ # If a selected/focused item is detached and later reattached, it stays
+ # selected/focused. That can give multiple selections even though
+ # selectmode=browse. Save and later restore the selection and focus as a
+ # workaround.
+ old_selection = _tree.selection()
+ old_focus = _tree.focus()
+
+ # Detach all tree items before re-stringing them. This is relatively fast,
+ # luckily.
+ _tree.detach(*_id_to_node.keys())
+
+ if _single_menu:
+ _build_menu_tree()
+ else:
+ _build_full_tree(_kconf.top_node)
+
+ _tree.selection_set(old_selection)
+ _tree.focus(old_focus)
+
+
+def _build_full_tree(menu):
+ # Updates the tree starting from menu.list, in full-tree mode. To speed
+ # things up, only open menus are updated. The menu-at-a-time logic here is
+ # to deal with invisible items that can show up outside show-all mode (see
+ # _shown_full_nodes()).
+
+ for node in _shown_full_nodes(menu):
+ _add_to_tree(node, _kconf.top_node)
+
+ # _shown_full_nodes() includes nodes from menus rooted at symbols, so
+ # we only need to check "real" menus/choices here
+ if node.list and not isinstance(node.item, Symbol):
+ if _tree.item(id(node), "open"):
+ _build_full_tree(node)
+ else:
+ # We're just probing here, so _shown_menu_nodes() will work
+ # fine, and might be a bit faster
+ shown = _shown_menu_nodes(node)
+ if shown:
+ # Dummy element to make the open/closed toggle appear
+ _tree.move(id(shown[0]), id(shown[0].parent), "end")
+
+
+def _shown_full_nodes(menu):
+ # Returns the list of menu nodes shown in 'menu' (a menu node for a menu)
+ # for full-tree mode. A tricky detail is that invisible items need to be
+ # shown if they have visible children.
+
+ def rec(node):
+ res = []
+
+ while node:
+ if _visible(node) or _show_all:
+ res.append(node)
+ if node.list and isinstance(node.item, Symbol):
+ # Nodes from menu created from dependencies
+ res += rec(node.list)
+
+ elif node.list and isinstance(node.item, Symbol):
+ # Show invisible symbols (defined with either 'config' and
+ # 'menuconfig') if they have visible children. This can happen
+ # for an m/y-valued symbol with an optional prompt
+ # ('prompt "foo" is COND') that is currently disabled.
+ shown_children = rec(node.list)
+ if shown_children:
+ res.append(node)
+ res += shown_children
+
+ node = node.next
+
+ return res
+
+ return rec(menu.list)
+
+
+def _build_menu_tree():
+ # Updates the tree in single-menu mode. See _build_full_tree() as well.
+
+ for node in _shown_menu_nodes(_cur_menu):
+ _add_to_tree(node, _cur_menu)
+
+
+def _shown_menu_nodes(menu):
+ # Used for single-menu mode. Similar to _shown_full_nodes(), but doesn't
+ # include children of symbols defined with 'menuconfig'.
+
+ def rec(node):
+ res = []
+
+ while node:
+ if _visible(node) or _show_all:
+ res.append(node)
+ if node.list and not node.is_menuconfig:
+ res += rec(node.list)
+
+ elif node.list and isinstance(node.item, Symbol):
+ shown_children = rec(node.list)
+ if shown_children:
+ # Invisible item with visible children
+ res.append(node)
+ if not node.is_menuconfig:
+ res += shown_children
+
+ node = node.next
+
+ return res
+
+ return rec(menu.list)
+
+
+def _visible(node):
+ # Returns True if the node should appear in the menu (outside show-all
+ # mode)
+
+ return node.prompt and expr_value(node.prompt[1]) and not \
+ (node.item == MENU and not expr_value(node.visibility))
+
+
+def _add_to_tree(node, top):
+ # Adds 'node' to the tree, at the end of its menu. We rely on going through
+ # the nodes linearly to get the correct order. 'top' holds the menu that
+ # corresponds to the top-level menu, and can vary in single-menu mode.
+
+ parent = node.parent
+ _tree.move(id(node), "" if parent is top else id(parent), "end")
+ _tree.item(
+ id(node),
+ text=_node_str(node),
+ # The _show_all test avoids showing invisible items in red outside
+ # show-all mode, which could look confusing/broken. Invisible symbols
+ # are shown outside show-all mode if an invisible symbol has visible
+ # children in an implicit menu.
+ tags=_img_tag(node) if _visible(node) or not _show_all else
+ _img_tag(node) + " invisible")
+
+
+def _node_str(node):
+ # Returns the string shown to the right of the image (if any) for the node
+
+ if node.prompt:
+ if node.item == COMMENT:
+ s = "*** {} ***".format(node.prompt[0])
+ else:
+ s = node.prompt[0]
+
+ if isinstance(node.item, Symbol):
+ sym = node.item
+
+ # Print "(NEW)" next to symbols without a user value (from e.g. a
+ # .config), but skip it for choice symbols in choices in y mode,
+ # and for symbols of UNKNOWN type (which generate a warning though)
+ if sym.user_value is None and sym.type and not \
+ (sym.choice and sym.choice.tri_value == 2):
+
+ s += " (NEW)"
+
+ elif isinstance(node.item, Symbol):
+ # Symbol without prompt (can show up in show-all)
+ s = "<{}>".format(node.item.name)
+
+ else:
+ # Choice without prompt. Use standard_sc_expr_str() so that it shows up
+ # as '<choice (name if any)>'.
+ s = standard_sc_expr_str(node.item)
+
+
+ if isinstance(node.item, Symbol):
+ sym = node.item
+ if sym.orig_type == STRING:
+ s += ": " + sym.str_value
+ elif sym.orig_type in (INT, HEX):
+ s = "({}) {}".format(sym.str_value, s)
+
+ elif isinstance(node.item, Choice) and node.item.tri_value == 2:
+ # Print the prompt of the selected symbol after the choice for
+ # choices in y mode
+ sym = node.item.selection
+ if sym:
+ for sym_node in sym.nodes:
+ # Use the prompt used at this choice location, in case the
+ # choice symbol is defined in multiple locations
+ if sym_node.parent is node and sym_node.prompt:
+ s += " ({})".format(sym_node.prompt[0])
+ break
+ else:
+ # If the symbol isn't defined at this choice location, then
+ # just use whatever prompt we can find for it
+ for sym_node in sym.nodes:
+ if sym_node.prompt:
+ s += " ({})".format(sym_node.prompt[0])
+ break
+
+ # In single-menu mode, print "--->" next to nodes that have menus that can
+ # potentially be entered. Print "----" if the menu is empty. We don't allow
+ # those to be entered.
+ if _single_menu and node.is_menuconfig:
+ s += " --->" if _shown_menu_nodes(node) else " ----"
+
+ return s
+
+
+def _img_tag(node):
+ # Returns the tag for the image that should be shown next to 'node', or the
+ # empty string if it shouldn't have an image
+
+ item = node.item
+
+ if item in (MENU, COMMENT) or not item.orig_type:
+ return ""
+
+ if item.orig_type in (STRING, INT, HEX):
+ return "edit"
+
+ # BOOL or TRISTATE
+
+ if _is_y_mode_choice_sym(item):
+ # Choice symbol in y-mode choice
+ return "selected" if item.choice.selection is item else "not-selected"
+
+ if len(item.assignable) <= 1:
+ # Pinned to a single value
+ return "" if isinstance(item, Choice) else item.str_value + "-locked"
+
+ if item.type == BOOL:
+ return item.str_value + "-bool"
+
+ # item.type == TRISTATE
+ if item.assignable == (1, 2):
+ return item.str_value + "-my"
+ return item.str_value + "-tri"
+
+
+def _is_y_mode_choice_sym(item):
+ # The choice mode is an upper bound on the visibility of choice symbols, so
+ # we can check the choice symbols' own visibility to see if the choice is
+ # in y mode
+ return isinstance(item, Symbol) and item.choice and item.visibility == 2
+
+
+def _tree_click(event):
+ # Click on the Kconfig Treeview
+
+ tree = event.widget
+ if tree.identify_element(event.x, event.y) == "image":
+ item = tree.identify_row(event.y)
+ # Select the item before possibly popping up a dialog for
+ # string/int/hex items, so that its help is visible
+ _select(tree, item)
+ _change_node(_id_to_node[item], tree.winfo_toplevel())
+ return "break"
+
+
+def _tree_double_click(event):
+ # Double-click on the Kconfig treeview
+
+ # Do an extra check to avoid weirdness when double-clicking in the tree
+ # heading area
+ if not _in_heading(event):
+ return _tree_enter(event)
+
+
+def _in_heading(event):
+ # Returns True if 'event' took place in the tree heading
+
+ tree = event.widget
+ return hasattr(tree, "identify_region") and \
+ tree.identify_region(event.x, event.y) in ("heading", "separator")
+
+
+def _tree_enter(event):
+ # Enter press or double-click within the Kconfig treeview. Prefer to
+ # open/close/enter menus, but toggle the value if that's not possible.
+
+ tree = event.widget
+ sel = tree.focus()
+ if sel:
+ node = _id_to_node[sel]
+
+ if tree.get_children(sel):
+ _tree_toggle_open(sel)
+ elif _single_menu_mode_menu(node, tree):
+ _enter_menu_and_select_first(node)
+ else:
+ _change_node(node, tree.winfo_toplevel())
+
+ return "break"
+
+
+def _tree_toggle(event):
+ # Space press within the Kconfig treeview. Prefer to toggle the value, but
+ # open/close/enter the menu if that's not possible.
+
+ tree = event.widget
+ sel = tree.focus()
+ if sel:
+ node = _id_to_node[sel]
+
+ if _changeable(node):
+ _change_node(node, tree.winfo_toplevel())
+ elif _single_menu_mode_menu(node, tree):
+ _enter_menu_and_select_first(node)
+ elif tree.get_children(sel):
+ _tree_toggle_open(sel)
+
+ return "break"
+
+
+def _tree_left_key(_):
+ # Left arrow key press within the Kconfig treeview
+
+ if _single_menu:
+ # Leave the current menu in single-menu mode
+ _leave_menu()
+ return "break"
+
+ # Otherwise, default action
+
+
+def _tree_right_key(_):
+ # Right arrow key press within the Kconfig treeview
+
+ sel = _tree.focus()
+ if sel:
+ node = _id_to_node[sel]
+ # If the node can be entered in single-menu mode, do it
+ if _single_menu_mode_menu(node, _tree):
+ _enter_menu_and_select_first(node)
+ return "break"
+
+ # Otherwise, default action
+
+
+def _single_menu_mode_menu(node, tree):
+ # Returns True if single-menu mode is on and 'node' is an (interface)
+ # menu that can be entered
+
+ return _single_menu and tree is _tree and node.is_menuconfig and \
+ _shown_menu_nodes(node)
+
+
+def _changeable(node):
+ # Returns True if 'node' is a Symbol/Choice whose value can be changed
+
+ sc = node.item
+
+ if not isinstance(sc, (Symbol, Choice)):
+ return False
+
+ # This will hit for invisible symbols, which appear in show-all mode and
+ # when an invisible symbol has visible children (which can happen e.g. for
+ # symbols with optional prompts)
+ if not (node.prompt and expr_value(node.prompt[1])):
+ return False
+
+ return sc.orig_type in (STRING, INT, HEX) or len(sc.assignable) > 1 \
+ or _is_y_mode_choice_sym(sc)
+
+
+def _tree_toggle_open(item):
+ # Opens/closes the Treeview item 'item'
+
+ if _tree.item(item, "open"):
+ _tree.item(item, open=False)
+ else:
+ node = _id_to_node[item]
+ if not isinstance(node.item, Symbol):
+ # Can only get here in full-tree mode
+ _build_full_tree(node)
+ _tree.item(item, open=True)
+
+
+def _tree_set_val(tri_val):
+ def tree_set_val(event):
+ # n/m/y press within the Kconfig treeview
+
+ # Sets the value of the currently selected item to 'tri_val', if that
+ # value can be assigned
+
+ sel = event.widget.focus()
+ if sel:
+ sc = _id_to_node[sel].item
+ if isinstance(sc, (Symbol, Choice)) and tri_val in sc.assignable:
+ _set_val(sc, tri_val)
+
+ return tree_set_val
+
+
+def _tree_open(_):
+ # Lazily populates the Kconfig tree when menus are opened in full-tree mode
+
+ if _single_menu:
+ # Work around https://core.tcl.tk/tk/tktview?name=368fa4561e
+ # ("ttk::treeview open/closed indicators can be toggled while hidden").
+ # Clicking on the hidden indicator will call _build_full_tree() in
+ # single-menu mode otherwise.
+ return
+
+ node = _id_to_node[_tree.focus()]
+ # _shown_full_nodes() includes nodes from menus rooted at symbols, so we
+ # only need to check "real" menus and choices here
+ if not isinstance(node.item, Symbol):
+ _build_full_tree(node)
+
+
+def _update_menu_path(_):
+ # Updates the displayed menu path when nodes are selected in the Kconfig
+ # treeview
+
+ sel = _tree.selection()
+ _menupath["text"] = _menu_path_info(_id_to_node[sel[0]]) if sel else ""
+
+
+def _item_row(item):
+ # Returns the row number 'item' appears on within the Kconfig treeview,
+ # starting from the top of the tree. Used to preserve scrolling.
+ #
+ # ttkTreeview.c in the Tk sources defines a RowNumber() function that does
+ # the same thing, but it's not exposed.
+
+ row = 0
+
+ while True:
+ prev = _tree.prev(item)
+ if prev:
+ item = prev
+ row += _n_rows(item)
+ else:
+ item = _tree.parent(item)
+ if not item:
+ return row
+ row += 1
+
+
+def _n_rows(item):
+ # _item_row() helper. Returns the number of rows occupied by 'item' and #
+ # its children.
+
+ rows = 1
+
+ if _tree.item(item, "open"):
+ for child in _tree.get_children(item):
+ rows += _n_rows(child)
+
+ return rows
+
+
+def _attached(item):
+ # Heuristic for checking if a Treeview item is attached. Doesn't seem to be
+ # good APIs for this. Might fail for super-obscure cases with tiny trees,
+ # but you'd just get a small scroll mess-up.
+
+ return bool(_tree.next(item) or _tree.prev(item) or _tree.parent(item))
+
+
+def _change_node(node, parent):
+ # Toggles/changes the value of 'node'. 'parent' is the parent window
+ # (either the main window or the jump-to dialog), in case we need to pop up
+ # a dialog.
+
+ if not _changeable(node):
+ return
+
+ # sc = symbol/choice
+ sc = node.item
+
+ if sc.type in (INT, HEX, STRING):
+ s = _set_val_dialog(node, parent)
+
+ # Tkinter can return 'unicode' strings on Python 2, which Kconfiglib
+ # can't deal with. UTF-8-encode the string to work around it.
+ if _PY2 and isinstance(s, unicode):
+ s = s.encode("utf-8", "ignore")
+
+ if s is not None:
+ _set_val(sc, s)
+
+ elif len(sc.assignable) == 1:
+ # Handles choice symbols for choices in y mode, which are a special
+ # case: .assignable can be (2,) while .tri_value is 0.
+ _set_val(sc, sc.assignable[0])
+
+ else:
+ # Set the symbol to the value after the current value in
+ # sc.assignable, with wrapping
+ val_index = sc.assignable.index(sc.tri_value)
+ _set_val(sc, sc.assignable[(val_index + 1) % len(sc.assignable)])
+
+
+def _set_val(sc, val):
+ # Wrapper around Symbol/Choice.set_value() for updating the menu state and
+ # _conf_changed
+
+ # Use the string representation of tristate values. This makes the format
+ # consistent for all symbol types.
+ if val in TRI_TO_STR:
+ val = TRI_TO_STR[val]
+
+ if val != sc.str_value:
+ sc.set_value(val)
+ _set_conf_changed(True)
+
+ # Update the tree and try to preserve the scroll. Do a cheaper variant
+ # than in the show-all case, that might mess up the scroll slightly in
+ # rare cases, but is fast and flicker-free.
+
+ stayput = _loc_ref_item() # Item to preserve scroll for
+ old_row = _item_row(stayput)
+
+ _update_tree()
+
+ # If the reference item disappeared (can happen if the change was done
+ # from the jump-to dialog), then avoid messing with the scroll and hope
+ # for the best
+ if _attached(stayput):
+ _tree.yview_scroll(_item_row(stayput) - old_row, "units")
+
+ if _jump_to_tree:
+ _update_jump_to_display()
+
+
+def _set_val_dialog(node, parent):
+ # Pops up a dialog for setting the value of the string/int/hex
+ # symbol at node 'node'. 'parent' is the parent window.
+
+ def ok(_=None):
+ # No 'nonlocal' in Python 2
+ global _entry_res
+
+ s = entry.get()
+ if sym.type == HEX and not s.startswith(("0x", "0X")):
+ s = "0x" + s
+
+ if _check_valid(dialog, entry, sym, s):
+ _entry_res = s
+ dialog.destroy()
+
+ def cancel(_=None):
+ global _entry_res
+ _entry_res = None
+ dialog.destroy()
+
+ sym = node.item
+
+ dialog = Toplevel(parent)
+ dialog.title("Enter {} value".format(TYPE_TO_STR[sym.type]))
+ dialog.resizable(False, False)
+ dialog.transient(parent)
+ dialog.protocol("WM_DELETE_WINDOW", cancel)
+
+ ttk.Label(dialog, text=node.prompt[0] + ":") \
+ .grid(column=0, row=0, columnspan=2, sticky="w", padx=".3c",
+ pady=".2c .05c")
+
+ entry = ttk.Entry(dialog, width=30)
+ # Start with the previous value in the editbox, selected
+ entry.insert(0, sym.str_value)
+ entry.selection_range(0, "end")
+ entry.grid(column=0, row=1, columnspan=2, sticky="ew", padx=".3c")
+ entry.focus_set()
+
+ range_info = _range_info(sym)
+ if range_info:
+ ttk.Label(dialog, text=range_info) \
+ .grid(column=0, row=2, columnspan=2, sticky="w", padx=".3c",
+ pady=".2c 0")
+
+ ttk.Button(dialog, text="OK", command=ok) \
+ .grid(column=0, row=4 if range_info else 3, sticky="e", padx=".3c",
+ pady=".4c")
+
+ ttk.Button(dialog, text="Cancel", command=cancel) \
+ .grid(column=1, row=4 if range_info else 3, padx="0 .3c")
+
+ # Give all horizontal space to the grid cell with the OK button, so that
+ # Cancel moves to the right
+ dialog.columnconfigure(0, weight=1)
+
+ _center_on_root(dialog)
+
+ # Hack to scroll the entry so that the end of the text is shown, from
+ # https://stackoverflow.com/questions/29334544/why-does-tkinters-entry-xview-moveto-fail.
+ # Related Tk ticket: https://core.tcl.tk/tk/info/2513186fff
+ def scroll_entry(_):
+ _root.update_idletasks()
+ entry.unbind("<Expose>")
+ entry.xview_moveto(1)
+ entry.bind("<Expose>", scroll_entry)
+
+ # The dialog must be visible before we can grab the input
+ dialog.wait_visibility()
+ dialog.grab_set()
+
+ dialog.bind("<Return>", ok)
+ dialog.bind("<KP_Enter>", ok)
+ dialog.bind("<Escape>", cancel)
+
+ # Wait for the user to be done with the dialog
+ parent.wait_window(dialog)
+
+ # Regrab the input in the parent
+ parent.grab_set()
+
+ return _entry_res
+
+
+def _center_on_root(dialog):
+ # Centers 'dialog' on the root window. It often ends up at some bad place
+ # like the top-left corner of the screen otherwise. See the menuconfig()
+ # function, which has similar logic.
+
+ dialog.withdraw()
+ _root.update_idletasks()
+
+ dialog_width = dialog.winfo_reqwidth()
+ dialog_height = dialog.winfo_reqheight()
+
+ screen_width = _root.winfo_screenwidth()
+ screen_height = _root.winfo_screenheight()
+
+ x = _root.winfo_rootx() + (_root.winfo_width() - dialog_width)//2
+ y = _root.winfo_rooty() + (_root.winfo_height() - dialog_height)//2
+
+ # Clamp so that no part of the dialog is outside the screen
+ if x + dialog_width > screen_width:
+ x = screen_width - dialog_width
+ elif x < 0:
+ x = 0
+ if y + dialog_height > screen_height:
+ y = screen_height - dialog_height
+ elif y < 0:
+ y = 0
+
+ dialog.geometry("+{}+{}".format(x, y))
+
+ dialog.deiconify()
+
+
+def _check_valid(dialog, entry, sym, s):
+ # Returns True if the string 's' is a well-formed value for 'sym'.
+ # Otherwise, pops up an error and returns False.
+
+ if sym.type not in (INT, HEX):
+ # Anything goes for non-int/hex symbols
+ return True
+
+ base = 10 if sym.type == INT else 16
+ try:
+ int(s, base)
+ except ValueError:
+ messagebox.showerror(
+ "Bad value",
+ "'{}' is a malformed {} value".format(
+ s, TYPE_TO_STR[sym.type]),
+ parent=dialog)
+ entry.focus_set()
+ return False
+
+ for low_sym, high_sym, cond in sym.ranges:
+ if expr_value(cond):
+ low_s = low_sym.str_value
+ high_s = high_sym.str_value
+
+ if not int(low_s, base) <= int(s, base) <= int(high_s, base):
+ messagebox.showerror(
+ "Value out of range",
+ "{} is outside the range {}-{}".format(s, low_s, high_s),
+ parent=dialog)
+ entry.focus_set()
+ return False
+
+ break
+
+ return True
+
+
+def _range_info(sym):
+ # Returns a string with information about the valid range for the symbol
+ # 'sym', or None if 'sym' doesn't have a range
+
+ if sym.type in (INT, HEX):
+ for low, high, cond in sym.ranges:
+ if expr_value(cond):
+ return "Range: {}-{}".format(low.str_value, high.str_value)
+
+ return None
+
+
+def _save(_=None):
+ # Tries to save the configuration
+
+ if _try_save(_kconf.write_config, _conf_filename, "configuration"):
+ _set_conf_changed(False)
+
+ _tree.focus_set()
+
+
+def _save_as():
+ # Pops up a dialog for saving the configuration to a specific location
+
+ global _conf_filename
+
+ filename = _conf_filename
+ while True:
+ filename = filedialog.asksaveasfilename(
+ title="Save configuration as",
+ initialdir=os.path.dirname(filename),
+ initialfile=os.path.basename(filename),
+ parent=_root)
+
+ if not filename:
+ break
+
+ if _try_save(_kconf.write_config, filename, "configuration"):
+ _conf_filename = filename
+ break
+
+ _tree.focus_set()
+
+
+def _save_minimal():
+ # Pops up a dialog for saving a minimal configuration (defconfig) to a
+ # specific location
+
+ global _minconf_filename
+
+ filename = _minconf_filename
+ while True:
+ filename = filedialog.asksaveasfilename(
+ title="Save minimal configuration as",
+ initialdir=os.path.dirname(filename),
+ initialfile=os.path.basename(filename),
+ parent=_root)
+
+ if not filename:
+ break
+
+ if _try_save(_kconf.write_min_config, filename,
+ "minimal configuration"):
+
+ _minconf_filename = filename
+ break
+
+ _tree.focus_set()
+
+
+def _open(_=None):
+ # Pops up a dialog for loading a configuration
+
+ global _conf_filename
+
+ if _conf_changed and \
+ not messagebox.askokcancel(
+ "Unsaved changes",
+ "You have unsaved changes. Load new configuration anyway?"):
+
+ return
+
+ filename = _conf_filename
+ while True:
+ filename = filedialog.askopenfilename(
+ title="Open configuration",
+ initialdir=os.path.dirname(filename),
+ initialfile=os.path.basename(filename),
+ parent=_root)
+
+ if not filename:
+ break
+
+ if _try_load(filename):
+ # Maybe something fancier could be done here later to try to
+ # preserve the scroll
+
+ _conf_filename = filename
+ _set_conf_changed(_needs_save())
+
+ if _single_menu and not _shown_menu_nodes(_cur_menu):
+ # Turn on show-all if we're in single-menu mode and would end
+ # up with an empty menu
+ _show_all_var.set(True)
+
+ _update_tree()
+
+ break
+
+ _tree.focus_set()
+
+
+def _toggle_showname(_):
+ # Toggles show-name mode on/off
+
+ _show_name_var.set(not _show_name_var.get())
+ _do_showname()
+
+
+def _do_showname():
+ # Updates the UI for the current show-name setting
+
+ # Columns do not automatically shrink/expand, so we have to update
+ # column widths ourselves
+
+ tree_width = _tree.winfo_width()
+
+ if _show_name_var.get():
+ _tree["displaycolumns"] = ("name",)
+ _tree["show"] = "tree headings"
+ name_width = tree_width//3
+ _tree.column("#0", width=max(tree_width - name_width, 1))
+ _tree.column("name", width=name_width)
+ else:
+ _tree["displaycolumns"] = ()
+ _tree["show"] = "tree"
+ _tree.column("#0", width=tree_width)
+
+ _tree.focus_set()
+
+
+def _toggle_showall(_):
+ # Toggles show-all mode on/off
+
+ _show_all_var.set(not _show_all)
+ _do_showall()
+
+
+def _do_showall():
+ # Updates the UI for the current show-all setting
+
+ # Don't allow turning off show-all if we'd end up with no visible nodes
+ if _nothing_shown():
+ _show_all_var.set(True)
+ return
+
+ # Save scroll information. old_scroll can end up negative here, if the
+ # reference item isn't shown (only invisible items on the screen, and
+ # show-all being turned off).
+
+ stayput = _vis_loc_ref_item()
+ # Probe the middle of the first row, to play it safe. identify_row(0) seems
+ # to return the row before the top row.
+ old_scroll = _item_row(stayput) - \
+ _item_row(_tree.identify_row(_treeview_rowheight//2))
+
+ _update_tree()
+
+ if _show_all:
+ # Deep magic: Unless we call update_idletasks(), the scroll adjustment
+ # below is restricted to the height of the old tree, instead of the
+ # height of the new tree. Since the tree with show-all on is guaranteed
+ # to be taller, and we want the maximum range, we only call it when
+ # turning show-all on.
+ #
+ # Strictly speaking, something similar ought to be done when changing
+ # symbol values, but it causes annoying flicker, and in 99% of cases
+ # things work anyway there (with usually minor scroll mess-ups in the
+ # 1% case).
+ _root.update_idletasks()
+
+ # Restore scroll
+ _tree.yview(_item_row(stayput) - old_scroll)
+
+ _tree.focus_set()
+
+
+def _nothing_shown():
+ # _do_showall() helper. Returns True if no nodes would get
+ # shown with the current show-all setting. Also handles the
+ # (obscure) case when there are no visible nodes in the entire
+ # tree, meaning guiconfig was automatically started in
+ # show-all mode, which mustn't be turned off.
+
+ return not _shown_menu_nodes(
+ _cur_menu if _single_menu else _kconf.top_node)
+
+
+def _toggle_tree_mode(_):
+ # Toggles single-menu mode on/off
+
+ _single_menu_var.set(not _single_menu)
+ _do_tree_mode()
+
+
+def _do_tree_mode():
+ # Updates the UI for the current tree mode (full-tree or single-menu)
+
+ loc_ref_node = _id_to_node[_loc_ref_item()]
+
+ if not _single_menu:
+ # _jump_to() -> _enter_menu() already updates the tree, but
+ # _jump_to() -> load_parents() doesn't, because it isn't always needed.
+ # We always need to update the tree here, e.g. to add/remove "--->".
+ _update_tree()
+
+ _jump_to(loc_ref_node)
+ _tree.focus_set()
+
+
+def _enter_menu_and_select_first(menu):
+ # Enters the menu 'menu' and selects the first item. Used in single-menu
+ # mode.
+
+ _enter_menu(menu)
+ _select(_tree, _tree.get_children()[0])
+
+
+def _enter_menu(menu):
+ # Enters the menu 'menu'. Used in single-menu mode.
+
+ global _cur_menu
+
+ _cur_menu = menu
+ _update_tree()
+
+ _backbutton["state"] = "disabled" if menu is _kconf.top_node else "normal"
+
+
+def _leave_menu():
+ # Leaves the current menu. Used in single-menu mode.
+
+ global _cur_menu
+
+ if _cur_menu is not _kconf.top_node:
+ old_menu = _cur_menu
+
+ _cur_menu = _parent_menu(_cur_menu)
+ _update_tree()
+
+ _select(_tree, id(old_menu))
+
+ if _cur_menu is _kconf.top_node:
+ _backbutton["state"] = "disabled"
+
+ _tree.focus_set()
+
+
+def _select(tree, item):
+ # Selects, focuses, and see()s 'item' in 'tree'
+
+ tree.selection_set(item)
+ tree.focus(item)
+ tree.see(item)
+
+
+def _loc_ref_item():
+ # Returns a Treeview item that can serve as a reference for the current
+ # scroll location. We try to make this item stay on the same row on the
+ # screen when updating the tree.
+
+ # If the selected item is visible, use that
+ sel = _tree.selection()
+ if sel and _tree.bbox(sel[0]):
+ return sel[0]
+
+ # Otherwise, use the middle item on the screen. If it doesn't exist, the
+ # tree is probably really small, so use the first item in the entire tree.
+ return _tree.identify_row(_tree.winfo_height()//2) or \
+ _tree.get_children()[0]
+
+
+def _vis_loc_ref_item():
+ # Like _loc_ref_item(), but finds a visible item around the reference item.
+ # Used when changing show-all mode, where non-visible (red) items will
+ # disappear.
+
+ item = _loc_ref_item()
+
+ vis_before = _vis_before(item)
+ if vis_before and _tree.bbox(vis_before):
+ return vis_before
+
+ vis_after = _vis_after(item)
+ if vis_after and _tree.bbox(vis_after):
+ return vis_after
+
+ return vis_before or vis_after
+
+
+def _vis_before(item):
+ # _vis_loc_ref_item() helper. Returns the first visible (not red) item,
+ # searching backwards from 'item'.
+
+ while item:
+ if not _tree.tag_has("invisible", item):
+ return item
+
+ prev = _tree.prev(item)
+ item = prev if prev else _tree.parent(item)
+
+ return None
+
+
+def _vis_after(item):
+ # _vis_loc_ref_item() helper. Returns the first visible (not red) item,
+ # searching forwards from 'item'.
+
+ while item:
+ if not _tree.tag_has("invisible", item):
+ return item
+
+ next = _tree.next(item)
+ if next:
+ item = next
+ else:
+ item = _tree.parent(item)
+ if not item:
+ break
+ item = _tree.next(item)
+
+ return None
+
+
+def _on_quit(_=None):
+ # Called when the user wants to exit
+
+ if not _conf_changed:
+ _quit("No changes to save (for '{}')".format(_conf_filename))
+ return
+
+ while True:
+ ync = messagebox.askyesnocancel("Quit", "Save changes?")
+ if ync is None:
+ return
+
+ if not ync:
+ _quit("Configuration ({}) was not saved".format(_conf_filename))
+ return
+
+ if _try_save(_kconf.write_config, _conf_filename, "configuration"):
+ # _try_save() already prints the "Configuration saved to ..."
+ # message
+ _quit()
+ return
+
+
+def _quit(msg=None):
+ # Quits the application
+
+ # Do not call sys.exit() here, in case we're being run from a script
+ _root.destroy()
+ if msg:
+ print(msg)
+
+
+def _try_save(save_fn, filename, description):
+ # Tries to save a configuration file. Pops up an error and returns False on
+ # failure.
+ #
+ # save_fn:
+ # Function to call with 'filename' to save the file
+ #
+ # description:
+ # String describing the thing being saved
+
+ try:
+ # save_fn() returns a message to print
+ msg = save_fn(filename)
+ _set_status(msg)
+ print(msg)
+ return True
+ except EnvironmentError as e:
+ messagebox.showerror(
+ "Error saving " + description,
+ "Error saving {} to '{}': {} (errno: {})"
+ .format(description, e.filename, e.strerror,
+ errno.errorcode[e.errno]))
+ return False
+
+
+def _try_load(filename):
+ # Tries to load a configuration file. Pops up an error and returns False on
+ # failure.
+ #
+ # filename:
+ # Configuration file to load
+
+ try:
+ msg = _kconf.load_config(filename)
+ _set_status(msg)
+ print(msg)
+ return True
+ except EnvironmentError as e:
+ messagebox.showerror(
+ "Error loading configuration",
+ "Error loading '{}': {} (errno: {})"
+ .format(filename, e.strerror, errno.errorcode[e.errno]))
+ return False
+
+
+def _jump_to_dialog(_=None):
+ # Pops up a dialog for jumping directly to a particular node. Symbol values
+ # can also be changed within the dialog.
+ #
+ # Note: There's nothing preventing this from doing an incremental search
+ # like menuconfig.py does, but currently it's a bit jerky for large Kconfig
+ # trees, at least when inputting the beginning of the search string. We'd
+ # need to somehow only update the tree items that are shown in the Treeview
+ # to fix it.
+
+ global _jump_to_tree
+
+ def search(_=None):
+ _update_jump_to_matches(msglabel, entry.get())
+
+ def jump_to_selected(event=None):
+ # Jumps to the selected node and closes the dialog
+
+ # Ignore double clicks on the image and in the heading area
+ if event and (tree.identify_element(event.x, event.y) == "image" or
+ _in_heading(event)):
+ return
+
+ sel = tree.selection()
+ if not sel:
+ return
+
+ node = _id_to_node[sel[0]]
+
+ if node not in _shown_menu_nodes(_parent_menu(node)):
+ _show_all_var.set(True)
+ if not _single_menu:
+ # See comment in _do_tree_mode()
+ _update_tree()
+
+ _jump_to(node)
+
+ dialog.destroy()
+
+ def tree_select(_):
+ jumpto_button["state"] = "normal" if tree.selection() else "disabled"
+
+
+ dialog = Toplevel(_root)
+ dialog.geometry("+{}+{}".format(
+ _root.winfo_rootx() + 50, _root.winfo_rooty() + 50))
+ dialog.title("Jump to symbol/choice/menu/comment")
+ dialog.minsize(128, 128) # See _create_ui()
+ dialog.transient(_root)
+
+ ttk.Label(dialog, text=_JUMP_TO_HELP) \
+ .grid(column=0, row=0, columnspan=2, sticky="w", padx=".1c",
+ pady=".1c")
+
+ entry = ttk.Entry(dialog)
+ entry.grid(column=0, row=1, sticky="ew", padx=".1c", pady=".1c")
+ entry.focus_set()
+
+ entry.bind("<Return>", search)
+ entry.bind("<KP_Enter>", search)
+
+ ttk.Button(dialog, text="Search", command=search) \
+ .grid(column=1, row=1, padx="0 .1c", pady="0 .1c")
+
+ msglabel = ttk.Label(dialog)
+ msglabel.grid(column=0, row=2, sticky="w", pady="0 .1c")
+
+ panedwindow, tree = _create_kconfig_tree_and_desc(dialog)
+ panedwindow.grid(column=0, row=3, columnspan=2, sticky="nsew")
+
+ # Clear tree
+ tree.set_children("")
+
+ _jump_to_tree = tree
+
+ jumpto_button = ttk.Button(dialog, text="Jump to selected item",
+ state="disabled", command=jump_to_selected)
+ jumpto_button.grid(column=0, row=4, columnspan=2, sticky="ns", pady=".1c")
+
+ dialog.columnconfigure(0, weight=1)
+ # Only the pane with the Kconfig tree and description grows vertically
+ dialog.rowconfigure(3, weight=1)
+
+ # See the menuconfig() function
+ _root.update_idletasks()
+ dialog.geometry(dialog.geometry())
+
+ # The dialog must be visible before we can grab the input
+ dialog.wait_visibility()
+ dialog.grab_set()
+
+ tree.bind("<Double-1>", jump_to_selected)
+ tree.bind("<Return>", jump_to_selected)
+ tree.bind("<KP_Enter>", jump_to_selected)
+ # add=True to avoid overriding the description text update
+ tree.bind("<<TreeviewSelect>>", tree_select, add=True)
+
+ dialog.bind("<Escape>", lambda _: dialog.destroy())
+
+ # Wait for the user to be done with the dialog
+ _root.wait_window(dialog)
+
+ _jump_to_tree = None
+
+ _tree.focus_set()
+
+
+def _update_jump_to_matches(msglabel, search_string):
+ # Searches for nodes matching the search string and updates
+ # _jump_to_matches. Puts a message in 'msglabel' if there are no matches,
+ # or regex errors.
+
+ global _jump_to_matches
+
+ _jump_to_tree.selection_set(())
+
+ try:
+ # We could use re.IGNORECASE here instead of lower(), but this is
+ # faster for regexes like '.*debug$' (though the '.*' is redundant
+ # there). Those probably have bad interactions with re.search(), which
+ # matches anywhere in the string.
+ regex_searches = [re.compile(regex).search
+ for regex in search_string.lower().split()]
+ except re.error as e:
+ msg = "Bad regular expression"
+ # re.error.msg was added in Python 3.5
+ if hasattr(e, "msg"):
+ msg += ": " + e.msg
+ msglabel["text"] = msg
+ # Clear tree
+ _jump_to_tree.set_children("")
+ return
+
+ _jump_to_matches = []
+ add_match = _jump_to_matches.append
+
+ for node in _sorted_sc_nodes():
+ # Symbol/choice
+ sc = node.item
+
+ for search in regex_searches:
+ # Both the name and the prompt might be missing, since
+ # we're searching both symbols and choices
+
+ # Does the regex match either the symbol name or the
+ # prompt (if any)?
+ if not (sc.name and search(sc.name.lower()) or
+ node.prompt and search(node.prompt[0].lower())):
+
+ # Give up on the first regex that doesn't match, to
+ # speed things up a bit when multiple regexes are
+ # entered
+ break
+
+ else:
+ add_match(node)
+
+ # Search menus and comments
+
+ for node in _sorted_menu_comment_nodes():
+ for search in regex_searches:
+ if not search(node.prompt[0].lower()):
+ break
+ else:
+ add_match(node)
+
+ msglabel["text"] = "" if _jump_to_matches else "No matches"
+
+ _update_jump_to_display()
+
+ if _jump_to_matches:
+ item = id(_jump_to_matches[0])
+ _jump_to_tree.selection_set(item)
+ _jump_to_tree.focus(item)
+
+
+def _update_jump_to_display():
+ # Updates the images and text for the items in _jump_to_matches, and sets
+ # them as the items of _jump_to_tree
+
+ # Micro-optimize a bit
+ item = _jump_to_tree.item
+ id_ = id
+ node_str = _node_str
+ img_tag = _img_tag
+ visible = _visible
+ for node in _jump_to_matches:
+ item(id_(node),
+ text=node_str(node),
+ tags=img_tag(node) if visible(node) else
+ img_tag(node) + " invisible")
+
+ _jump_to_tree.set_children("", *map(id, _jump_to_matches))
+
+
+def _jump_to(node):
+ # Jumps directly to 'node' and selects it
+
+ if _single_menu:
+ _enter_menu(_parent_menu(node))
+ else:
+ _load_parents(node)
+
+ _select(_tree, id(node))
+
+
+# Obscure Python: We never pass a value for cached_nodes, and it keeps pointing
+# to the same list. This avoids a global.
+def _sorted_sc_nodes(cached_nodes=[]):
+ # Returns a sorted list of symbol and choice nodes to search. The symbol
+ # nodes appear first, sorted by name, and then the choice nodes, sorted by
+ # prompt and (secondarily) name.
+
+ if not cached_nodes:
+ # Add symbol nodes
+ for sym in sorted(_kconf.unique_defined_syms,
+ key=lambda sym: sym.name):
+ # += is in-place for lists
+ cached_nodes += sym.nodes
+
+ # Add choice nodes
+
+ choices = sorted(_kconf.unique_choices,
+ key=lambda choice: choice.name or "")
+
+ cached_nodes += sorted(
+ [node for choice in choices for node in choice.nodes],
+ key=lambda node: node.prompt[0] if node.prompt else "")
+
+ return cached_nodes
+
+
+def _sorted_menu_comment_nodes(cached_nodes=[]):
+ # Returns a list of menu and comment nodes to search, sorted by prompt,
+ # with the menus first
+
+ if not cached_nodes:
+ def prompt_text(mc):
+ return mc.prompt[0]
+
+ cached_nodes += sorted(_kconf.menus, key=prompt_text)
+ cached_nodes += sorted(_kconf.comments, key=prompt_text)
+
+ return cached_nodes
+
+
+def _load_parents(node):
+ # Menus are lazily populated as they're opened in full-tree mode, but
+ # jumping to an item needs its parent menus to be populated. This function
+ # populates 'node's parents.
+
+ # Get all parents leading up to 'node', sorted with the root first
+ parents = []
+ cur = node.parent
+ while cur is not _kconf.top_node:
+ parents.append(cur)
+ cur = cur.parent
+ parents.reverse()
+
+ for i, parent in enumerate(parents):
+ if not _tree.item(id(parent), "open"):
+ # Found a closed menu. Populate it and all the remaining menus
+ # leading up to 'node'.
+ for parent in parents[i:]:
+ # We only need to populate "real" menus/choices. Implicit menus
+ # are populated when their parents menus are entered.
+ if not isinstance(parent.item, Symbol):
+ _build_full_tree(parent)
+ return
+
+
+def _parent_menu(node):
+ # Returns the menu node of the menu that contains 'node'. In addition to
+ # proper 'menu's, this might also be a 'menuconfig' symbol or a 'choice'.
+ # "Menu" here means a menu in the interface.
+
+ menu = node.parent
+ while not menu.is_menuconfig:
+ menu = menu.parent
+ return menu
+
+
+def _trace_write(var, fn):
+ # Makes fn() be called whenever the Tkinter Variable 'var' changes value
+
+ # trace_variable() is deprecated according to the docstring,
+ # which recommends trace_add()
+ if hasattr(var, "trace_add"):
+ var.trace_add("write", fn)
+ else:
+ var.trace_variable("w", fn)
+
+
+def _info_str(node):
+ # Returns information about the menu node 'node' as a string.
+ #
+ # The helper functions are responsible for adding newlines. This allows
+ # them to return "" if they don't want to add any output.
+
+ if isinstance(node.item, Symbol):
+ sym = node.item
+
+ return (
+ _name_info(sym) +
+ _help_info(sym) +
+ _direct_dep_info(sym) +
+ _defaults_info(sym) +
+ _select_imply_info(sym) +
+ _kconfig_def_info(sym)
+ )
+
+ if isinstance(node.item, Choice):
+ choice = node.item
+
+ return (
+ _name_info(choice) +
+ _help_info(choice) +
+ 'Mode: {}\n\n'.format(choice.str_value) +
+ _choice_syms_info(choice) +
+ _direct_dep_info(choice) +
+ _defaults_info(choice) +
+ _kconfig_def_info(choice)
+ )
+
+ # node.item in (MENU, COMMENT)
+ return _kconfig_def_info(node)
+
+
+def _name_info(sc):
+ # Returns a string with the name of the symbol/choice. Choices are shown as
+ # <choice (name if any)>.
+
+ return (sc.name if sc.name else standard_sc_expr_str(sc)) + "\n\n"
+
+
+def _value_info(sym):
+ # Returns a string showing 'sym's value
+
+ # Only put quotes around the value for string symbols
+ return "Value: {}\n".format(
+ '"{}"'.format(sym.str_value)
+ if sym.orig_type == STRING
+ else sym.str_value)
+
+
+def _choice_syms_info(choice):
+ # Returns a string listing the choice symbols in 'choice'. Adds
+ # "(selected)" next to the selected one.
+
+ s = "Choice symbols:\n"
+
+ for sym in choice.syms:
+ s += " - " + sym.name
+ if sym is choice.selection:
+ s += " (selected)"
+ s += "\n"
+
+ return s + "\n"
+
+
+def _help_info(sc):
+ # Returns a string with the help text(s) of 'sc' (Symbol or Choice).
+ # Symbols and choices defined in multiple locations can have multiple help
+ # texts.
+
+ s = ""
+
+ for node in sc.nodes:
+ if node.help is not None:
+ s += node.help + "\n\n"
+
+ return s
+
+
+def _direct_dep_info(sc):
+ # Returns a string describing the direct dependencies of 'sc' (Symbol or
+ # Choice). The direct dependencies are the OR of the dependencies from each
+ # definition location. The dependencies at each definition location come
+ # from 'depends on' and dependencies inherited from parent items.
+
+ return "" if sc.direct_dep is _kconf.y else \
+ 'Direct dependencies (={}):\n{}\n' \
+ .format(TRI_TO_STR[expr_value(sc.direct_dep)],
+ _split_expr_info(sc.direct_dep, 2))
+
+
+def _defaults_info(sc):
+ # Returns a string describing the defaults of 'sc' (Symbol or Choice)
+
+ if not sc.defaults:
+ return ""
+
+ s = "Default"
+ if len(sc.defaults) > 1:
+ s += "s"
+ s += ":\n"
+
+ for val, cond in sc.orig_defaults:
+ s += " - "
+ if isinstance(sc, Symbol):
+ s += _expr_str(val)
+
+ # Skip the tristate value hint if the expression is just a single
+ # symbol. _expr_str() already shows its value as a string.
+ #
+ # This also avoids showing the tristate value for string/int/hex
+ # defaults, which wouldn't make any sense.
+ if isinstance(val, tuple):
+ s += ' (={})'.format(TRI_TO_STR[expr_value(val)])
+ else:
+ # Don't print the value next to the symbol name for choice
+ # defaults, as it looks a bit confusing
+ s += val.name
+ s += "\n"
+
+ if cond is not _kconf.y:
+ s += " Condition (={}):\n{}" \
+ .format(TRI_TO_STR[expr_value(cond)],
+ _split_expr_info(cond, 4))
+
+ return s + "\n"
+
+
+def _split_expr_info(expr, indent):
+ # Returns a string with 'expr' split into its top-level && or || operands,
+ # with one operand per line, together with the operand's value. This is
+ # usually enough to get something readable for long expressions. A fancier
+ # recursive thingy would be possible too.
+ #
+ # indent:
+ # Number of leading spaces to add before the split expression.
+
+ if len(split_expr(expr, AND)) > 1:
+ split_op = AND
+ op_str = "&&"
+ else:
+ split_op = OR
+ op_str = "||"
+
+ s = ""
+ for i, term in enumerate(split_expr(expr, split_op)):
+ s += "{}{} {}".format(indent*" ",
+ " " if i == 0 else op_str,
+ _expr_str(term))
+
+ # Don't bother showing the value hint if the expression is just a
+ # single symbol. _expr_str() already shows its value.
+ if isinstance(term, tuple):
+ s += " (={})".format(TRI_TO_STR[expr_value(term)])
+
+ s += "\n"
+
+ return s
+
+
+def _select_imply_info(sym):
+ # Returns a string with information about which symbols 'select' or 'imply'
+ # 'sym'. The selecting/implying symbols are grouped according to which
+ # value they select/imply 'sym' to (n/m/y).
+
+ def sis(expr, val, title):
+ # sis = selects/implies
+ sis = [si for si in split_expr(expr, OR) if expr_value(si) == val]
+ if not sis:
+ return ""
+
+ res = title
+ for si in sis:
+ res += " - {}\n".format(split_expr(si, AND)[0].name)
+ return res + "\n"
+
+ s = ""
+
+ if sym.rev_dep is not _kconf.n:
+ s += sis(sym.rev_dep, 2,
+ "Symbols currently y-selecting this symbol:\n")
+ s += sis(sym.rev_dep, 1,
+ "Symbols currently m-selecting this symbol:\n")
+ s += sis(sym.rev_dep, 0,
+ "Symbols currently n-selecting this symbol (no effect):\n")
+
+ if sym.weak_rev_dep is not _kconf.n:
+ s += sis(sym.weak_rev_dep, 2,
+ "Symbols currently y-implying this symbol:\n")
+ s += sis(sym.weak_rev_dep, 1,
+ "Symbols currently m-implying this symbol:\n")
+ s += sis(sym.weak_rev_dep, 0,
+ "Symbols currently n-implying this symbol (no effect):\n")
+
+ return s
+
+
+def _kconfig_def_info(item):
+ # Returns a string with the definition of 'item' in Kconfig syntax,
+ # together with the definition location(s) and their include and menu paths
+
+ nodes = [item] if isinstance(item, MenuNode) else item.nodes
+
+ s = "Kconfig definition{}, with parent deps. propagated to 'depends on'\n" \
+ .format("s" if len(nodes) > 1 else "")
+ s += (len(s) - 1)*"="
+
+ for node in nodes:
+ s += "\n\n" \
+ "At {}:{}\n" \
+ "{}" \
+ "Menu path: {}\n\n" \
+ "{}" \
+ .format(node.filename, node.linenr,
+ _include_path_info(node),
+ _menu_path_info(node),
+ node.custom_str(_name_and_val_str))
+
+ return s
+
+
+def _include_path_info(node):
+ if not node.include_path:
+ # In the top-level Kconfig file
+ return ""
+
+ return "Included via {}\n".format(
+ " -> ".join("{}:{}".format(filename, linenr)
+ for filename, linenr in node.include_path))
+
+
+def _menu_path_info(node):
+ # Returns a string describing the menu path leading up to 'node'
+
+ path = ""
+
+ while node.parent is not _kconf.top_node:
+ node = node.parent
+
+ # Promptless choices might appear among the parents. Use
+ # standard_sc_expr_str() for them, so that they show up as
+ # '<choice (name if any)>'.
+ path = " -> " + (node.prompt[0] if node.prompt else
+ standard_sc_expr_str(node.item)) + path
+
+ return "(Top)" + path
+
+
+def _name_and_val_str(sc):
+ # Custom symbol/choice printer that shows symbol values after symbols
+
+ # Show the values of non-constant (non-quoted) symbols that don't look like
+ # numbers. Things like 123 are actually symbol references, and only work as
+ # expected due to undefined symbols getting their name as their value.
+ # Showing the symbol value for those isn't helpful though.
+ if isinstance(sc, Symbol) and not sc.is_constant and not _is_num(sc.name):
+ if not sc.nodes:
+ # Undefined symbol reference
+ return "{}(undefined/n)".format(sc.name)
+
+ return '{}(={})'.format(sc.name, sc.str_value)
+
+ # For other items, use the standard format
+ return standard_sc_expr_str(sc)
+
+
+def _expr_str(expr):
+ # Custom expression printer that shows symbol values
+ return expr_str(expr, _name_and_val_str)
+
+
+def _is_num(name):
+ # Heuristic to see if a symbol name looks like a number, for nicer output
+ # when printing expressions. Things like 16 are actually symbol names, only
+ # they get their name as their value when the symbol is undefined.
+
+ try:
+ int(name)
+ except ValueError:
+ if not name.startswith(("0x", "0X")):
+ return False
+
+ try:
+ int(name, 16)
+ except ValueError:
+ return False
+
+ return True
+
+
+if __name__ == "__main__":
+ _main()
diff --git a/ext/Kconfiglib/import/kconfiglib.py b/ext/Kconfiglib/import/kconfiglib.py
new file mode 100644
index 0000000..e5c2dcc
--- /dev/null
+++ b/ext/Kconfiglib/import/kconfiglib.py
@@ -0,0 +1,7192 @@
+# Copyright (c) 2011-2019, Ulf Magnusson
+# SPDX-License-Identifier: ISC
+
+"""
+Overview
+========
+
+Kconfiglib is a Python 2/3 library for scripting and extracting information
+from Kconfig (https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt)
+configuration systems.
+
+See the homepage at https://github.com/ulfalizer/Kconfiglib for a longer
+overview.
+
+Since Kconfiglib 12.0.0, the library version is available in
+kconfiglib.VERSION, which is a (<major>, <minor>, <patch>) tuple, e.g.
+(12, 0, 0).
+
+
+Using Kconfiglib on the Linux kernel with the Makefile targets
+==============================================================
+
+For the Linux kernel, a handy interface is provided by the
+scripts/kconfig/Makefile patch, which can be applied with either 'git am' or
+the 'patch' utility:
+
+ $ wget -qO- https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch | git am
+ $ wget -qO- https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch | patch -p1
+
+Warning: Not passing -p1 to patch will cause the wrong file to be patched.
+
+Please tell me if the patch does not apply. It should be trivial to apply
+manually, as it's just a block of text that needs to be inserted near the other
+*conf: targets in scripts/kconfig/Makefile.
+
+Look further down for a motivation for the Makefile patch and for instructions
+on how you can use Kconfiglib without it.
+
+If you do not wish to install Kconfiglib via pip, the Makefile patch is set up
+so that you can also just clone Kconfiglib into the kernel root:
+
+ $ git clone git://github.com/ulfalizer/Kconfiglib.git
+ $ git am Kconfiglib/makefile.patch (or 'patch -p1 < Kconfiglib/makefile.patch')
+
+Warning: The directory name Kconfiglib/ is significant in this case, because
+it's added to PYTHONPATH by the new targets in makefile.patch.
+
+The targets added by the Makefile patch are described in the following
+sections.
+
+
+make kmenuconfig
+----------------
+
+This target runs the curses menuconfig interface with Python 3. As of
+Kconfiglib 12.2.0, both Python 2 and Python 3 are supported (previously, only
+Python 3 was supported, so this was a backport).
+
+
+make guiconfig
+--------------
+
+This target runs the Tkinter menuconfig interface. Both Python 2 and Python 3
+are supported. To change the Python interpreter used, pass
+PYTHONCMD=<executable> to 'make'. The default is 'python'.
+
+
+make [ARCH=<arch>] iscriptconfig
+--------------------------------
+
+This target gives an interactive Python prompt where a Kconfig instance has
+been preloaded and is available in 'kconf'. To change the Python interpreter
+used, pass PYTHONCMD=<executable> to 'make'. The default is 'python'.
+
+To get a feel for the API, try evaluating and printing the symbols in
+kconf.defined_syms, and explore the MenuNode menu tree starting at
+kconf.top_node by following 'next' and 'list' pointers.
+
+The item contained in a menu node is found in MenuNode.item (note that this can
+be one of the constants kconfiglib.MENU and kconfiglib.COMMENT), and all
+symbols and choices have a 'nodes' attribute containing their menu nodes
+(usually only one). Printing a menu node will print its item, in Kconfig
+format.
+
+If you want to look up a symbol by name, use the kconf.syms dictionary.
+
+
+make scriptconfig SCRIPT=<script> [SCRIPT_ARG=<arg>]
+----------------------------------------------------
+
+This target runs the Python script given by the SCRIPT parameter on the
+configuration. sys.argv[1] holds the name of the top-level Kconfig file
+(currently always "Kconfig" in practice), and sys.argv[2] holds the SCRIPT_ARG
+argument, if given.
+
+See the examples/ subdirectory for example scripts.
+
+
+make dumpvarsconfig
+-------------------
+
+This target prints a list of all environment variables referenced from the
+Kconfig files, together with their values. See the
+Kconfiglib/examples/dumpvars.py script.
+
+Only environment variables that are referenced via the Kconfig preprocessor
+$(FOO) syntax are included. The preprocessor was added in Linux 4.18.
+
+
+Using Kconfiglib without the Makefile targets
+=============================================
+
+The make targets are only needed to pick up environment variables exported from
+the Kbuild makefiles and referenced inside Kconfig files, via e.g.
+'source "arch/$(SRCARCH)/Kconfig" and commands run via '$(shell,...)'.
+
+These variables are referenced as of writing (Linux 4.18), together with sample
+values:
+
+ srctree (.)
+ ARCH (x86)
+ SRCARCH (x86)
+ KERNELVERSION (4.18.0)
+ CC (gcc)
+ HOSTCC (gcc)
+ HOSTCXX (g++)
+ CC_VERSION_TEXT (gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0)
+
+Older kernels only reference ARCH, SRCARCH, and KERNELVERSION.
+
+If your kernel is recent enough (4.18+), you can get a list of referenced
+environment variables via 'make dumpvarsconfig' (see above). Note that this
+command is added by the Makefile patch.
+
+To run Kconfiglib without the Makefile patch, set the environment variables
+manually:
+
+ $ srctree=. ARCH=x86 SRCARCH=x86 KERNELVERSION=`make kernelversion` ... python(3)
+ >>> import kconfiglib
+ >>> kconf = kconfiglib.Kconfig() # filename defaults to "Kconfig"
+
+Search the top-level Makefile for "Additional ARCH settings" to see other
+possibilities for ARCH and SRCARCH.
+
+
+Intro to symbol values
+======================
+
+Kconfiglib has the same assignment semantics as the C implementation.
+
+Any symbol can be assigned a value by the user (via Kconfig.load_config() or
+Symbol.set_value()), but this user value is only respected if the symbol is
+visible, which corresponds to it (currently) being visible in the menuconfig
+interface.
+
+For symbols with prompts, the visibility of the symbol is determined by the
+condition on the prompt. Symbols without prompts are never visible, so setting
+a user value on them is pointless. A warning will be printed by default if
+Symbol.set_value() is called on a promptless symbol. Assignments to promptless
+symbols are normal within a .config file, so no similar warning will be printed
+by load_config().
+
+Dependencies from parents and 'if'/'depends on' are propagated to properties,
+including prompts, so these two configurations are logically equivalent:
+
+(1)
+
+ menu "menu"
+ depends on A
+
+ if B
+
+ config FOO
+ tristate "foo" if D
+ default y
+ depends on C
+
+ endif
+
+ endmenu
+
+(2)
+
+ menu "menu"
+ depends on A
+
+ config FOO
+ tristate "foo" if A && B && C && D
+ default y if A && B && C
+
+ endmenu
+
+In this example, A && B && C && D (the prompt condition) needs to be non-n for
+FOO to be visible (assignable). If its value is m, the symbol can only be
+assigned the value m: The visibility sets an upper bound on the value that can
+be assigned by the user, and any higher user value will be truncated down.
+
+'default' properties are independent of the visibility, though a 'default' will
+often get the same condition as the prompt due to dependency propagation.
+'default' properties are used if the symbol is not visible or has no user
+value.
+
+Symbols with no user value (or that have a user value but are not visible) and
+no (active) 'default' default to n for bool/tristate symbols, and to the empty
+string for other symbol types.
+
+'select' works similarly to symbol visibility, but sets a lower bound on the
+value of the symbol. The lower bound is determined by the value of the
+select*ing* symbol. 'select' does not respect visibility, so non-visible
+symbols can be forced to a particular (minimum) value by a select as well.
+
+For non-bool/tristate symbols, it only matters whether the visibility is n or
+non-n: m visibility acts the same as y visibility.
+
+Conditions on 'default' and 'select' work in mostly intuitive ways. If the
+condition is n, the 'default' or 'select' is disabled. If it is m, the
+'default' or 'select' value (the value of the selecting symbol) is truncated
+down to m.
+
+When writing a configuration with Kconfig.write_config(), only symbols that are
+visible, have an (active) default, or are selected will get written out (note
+that this includes all symbols that would accept user values). Kconfiglib
+matches the .config format produced by the C implementations down to the
+character. This eases testing.
+
+For a visible bool/tristate symbol FOO with value n, this line is written to
+.config:
+
+ # CONFIG_FOO is not set
+
+The point is to remember the user n selection (which might differ from the
+default value the symbol would get), while at the same sticking to the rule
+that undefined corresponds to n (.config uses Makefile format, making the line
+above a comment). When the .config file is read back in, this line will be
+treated the same as the following assignment:
+
+ CONFIG_FOO=n
+
+In Kconfiglib, the set of (currently) assignable values for a bool/tristate
+symbol appear in Symbol.assignable. For other symbol types, just check if
+sym.visibility is non-0 (non-n) to see whether the user value will have an
+effect.
+
+
+Intro to the menu tree
+======================
+
+The menu structure, as seen in e.g. menuconfig, is represented by a tree of
+MenuNode objects. The top node of the configuration corresponds to an implicit
+top-level menu, the title of which is shown at the top in the standard
+menuconfig interface. (The title is also available in Kconfig.mainmenu_text in
+Kconfiglib.)
+
+The top node is found in Kconfig.top_node. From there, you can visit child menu
+nodes by following the 'list' pointer, and any following menu nodes by
+following the 'next' pointer. Usually, a non-None 'list' pointer indicates a
+menu or Choice, but menu nodes for symbols can sometimes have a non-None 'list'
+pointer too due to submenus created implicitly from dependencies.
+
+MenuNode.item is either a Symbol or a Choice object, or one of the constants
+MENU and COMMENT. The prompt of the menu node can be found in MenuNode.prompt,
+which also holds the title for menus and comments. For Symbol and Choice,
+MenuNode.help holds the help text (if any, otherwise None).
+
+Most symbols will only have a single menu node. A symbol defined in multiple
+locations will have one menu node for each location. The list of menu nodes for
+a Symbol or Choice can be found in the Symbol/Choice.nodes attribute.
+
+Note that prompts and help texts for symbols and choices are stored in their
+menu node(s) rather than in the Symbol or Choice objects themselves. This makes
+it possible to define a symbol in multiple locations with a different prompt or
+help text in each location. To get the help text or prompt for a symbol with a
+single menu node, do sym.nodes[0].help and sym.nodes[0].prompt, respectively.
+The prompt is a (text, condition) tuple, where condition determines the
+visibility (see 'Intro to expressions' below).
+
+This organization mirrors the C implementation. MenuNode is called
+'struct menu' there, but I thought "menu" was a confusing name.
+
+It is possible to give a Choice a name and define it in multiple locations,
+hence why Choice.nodes is also a list.
+
+As a convenience, the properties added at a particular definition location are
+available on the MenuNode itself, in e.g. MenuNode.defaults. This is helpful
+when generating documentation, so that symbols/choices defined in multiple
+locations can be shown with the correct properties at each location.
+
+
+Intro to expressions
+====================
+
+Expressions can be evaluated with the expr_value() function and printed with
+the expr_str() function (these are used internally as well). Evaluating an
+expression always yields a tristate value, where n, m, and y are represented as
+0, 1, and 2, respectively.
+
+The following table should help you figure out how expressions are represented.
+A, B, C, ... are symbols (Symbol instances), NOT is the kconfiglib.NOT
+constant, etc.
+
+Expression Representation
+---------- --------------
+A A
+"A" A (constant symbol)
+!A (NOT, A)
+A && B (AND, A, B)
+A && B && C (AND, A, (AND, B, C))
+A || B (OR, A, B)
+A || (B && C && D) (OR, A, (AND, B, (AND, C, D)))
+A = B (EQUAL, A, B)
+A != "foo" (UNEQUAL, A, foo (constant symbol))
+A && B = C && D (AND, A, (AND, (EQUAL, B, C), D))
+n Kconfig.n (constant symbol)
+m Kconfig.m (constant symbol)
+y Kconfig.y (constant symbol)
+"y" Kconfig.y (constant symbol)
+
+Strings like "foo" in 'default "foo"' or 'depends on SYM = "foo"' are
+represented as constant symbols, so the only values that appear in expressions
+are symbols***. This mirrors the C implementation.
+
+***For choice symbols, the parent Choice will appear in expressions as well,
+but it's usually invisible as the value interfaces of Symbol and Choice are
+identical. This mirrors the C implementation and makes different choice modes
+"just work".
+
+Manual evaluation examples:
+
+ - The value of A && B is min(A.tri_value, B.tri_value)
+
+ - The value of A || B is max(A.tri_value, B.tri_value)
+
+ - The value of !A is 2 - A.tri_value
+
+ - The value of A = B is 2 (y) if A.str_value == B.str_value, and 0 (n)
+ otherwise. Note that str_value is used here instead of tri_value.
+
+ For constant (as well as undefined) symbols, str_value matches the name of
+ the symbol. This mirrors the C implementation and explains why
+ 'depends on SYM = "foo"' above works as expected.
+
+n/m/y are automatically converted to the corresponding constant symbols
+"n"/"m"/"y" (Kconfig.n/m/y) during parsing.
+
+Kconfig.const_syms is a dictionary like Kconfig.syms but for constant symbols.
+
+If a condition is missing (e.g., <cond> when the 'if <cond>' is removed from
+'default A if <cond>'), it is actually Kconfig.y. The standard __str__()
+functions just avoid printing 'if y' conditions to give cleaner output.
+
+
+Kconfig extensions
+==================
+
+Kconfiglib includes a couple of Kconfig extensions:
+
+'source' with relative path
+---------------------------
+
+The 'rsource' statement sources Kconfig files with a path relative to directory
+of the Kconfig file containing the 'rsource' statement, instead of relative to
+the project root.
+
+Consider following directory tree:
+
+ Project
+ +--Kconfig
+ |
+ +--src
+ +--Kconfig
+ |
+ +--SubSystem1
+ +--Kconfig
+ |
+ +--ModuleA
+ +--Kconfig
+
+In this example, assume that src/SubSystem1/Kconfig wants to source
+src/SubSystem1/ModuleA/Kconfig.
+
+With 'source', this statement would be used:
+
+ source "src/SubSystem1/ModuleA/Kconfig"
+
+With 'rsource', this turns into
+
+ rsource "ModuleA/Kconfig"
+
+If an absolute path is given to 'rsource', it acts the same as 'source'.
+
+'rsource' can be used to create "position-independent" Kconfig trees that can
+be moved around freely.
+
+
+Globbing 'source'
+-----------------
+
+'source' and 'rsource' accept glob patterns, sourcing all matching Kconfig
+files. They require at least one matching file, raising a KconfigError
+otherwise.
+
+For example, the following statement might source sub1/foofoofoo and
+sub2/foobarfoo:
+
+ source "sub[12]/foo*foo"
+
+The glob patterns accepted are the same as for the standard glob.glob()
+function.
+
+Two additional statements are provided for cases where it's acceptable for a
+pattern to match no files: 'osource' and 'orsource' (the o is for "optional").
+
+For example, the following statements will be no-ops if neither "foo" nor any
+files matching "bar*" exist:
+
+ osource "foo"
+ osource "bar*"
+
+'orsource' does a relative optional source.
+
+'source' and 'osource' are analogous to 'include' and '-include' in Make.
+
+
+Generalized def_* keywords
+--------------------------
+
+def_int, def_hex, and def_string are available in addition to def_bool and
+def_tristate, allowing int, hex, and string symbols to be given a type and a
+default at the same time.
+
+
+Extra optional warnings
+-----------------------
+
+Some optional warnings can be controlled via environment variables:
+
+ - KCONFIG_WARN_UNDEF: If set to 'y', warnings will be generated for all
+ references to undefined symbols within Kconfig files. The only gotcha is
+ that all hex literals must be prefixed with "0x" or "0X", to make it
+ possible to distinguish them from symbol references.
+
+ Some projects (e.g. the Linux kernel) use multiple Kconfig trees with many
+ shared Kconfig files, leading to some safe undefined symbol references.
+ KCONFIG_WARN_UNDEF is useful in projects that only have a single Kconfig
+ tree though.
+
+ KCONFIG_STRICT is an older alias for this environment variable, supported
+ for backwards compatibility.
+
+ - KCONFIG_WARN_UNDEF_ASSIGN: If set to 'y', warnings will be generated for
+ all assignments to undefined symbols within .config files. By default, no
+ such warnings are generated.
+
+ This warning can also be enabled/disabled via the Kconfig.warn_assign_undef
+ variable.
+
+
+Preprocessor user functions defined in Python
+---------------------------------------------
+
+Preprocessor functions can be defined in Python, which makes it simple to
+integrate information from existing Python tools into Kconfig (e.g. to have
+Kconfig symbols depend on hardware information stored in some other format).
+
+Putting a Python module named kconfigfunctions(.py) anywhere in sys.path will
+cause it to be imported by Kconfiglib (in Kconfig.__init__()). Note that
+sys.path can be customized via PYTHONPATH, and includes the directory of the
+module being run by default, as well as installation directories.
+
+If the KCONFIG_FUNCTIONS environment variable is set, it gives a different
+module name to use instead of 'kconfigfunctions'.
+
+The imported module is expected to define a global dictionary named 'functions'
+that maps function names to Python functions, as follows:
+
+ def my_fn(kconf, name, arg_1, arg_2, ...):
+ # kconf:
+ # Kconfig instance
+ #
+ # name:
+ # Name of the user-defined function ("my-fn"). Think argv[0].
+ #
+ # arg_1, arg_2, ...:
+ # Arguments passed to the function from Kconfig (strings)
+ #
+ # Returns a string to be substituted as the result of calling the
+ # function
+ ...
+
+ def my_other_fn(kconf, name, arg_1, arg_2, ...):
+ ...
+
+ functions = {
+ "my-fn": (my_fn, <min.args>, <max.args>/None),
+ "my-other-fn": (my_other_fn, <min.args>, <max.args>/None),
+ ...
+ }
+
+ ...
+
+<min.args> and <max.args> are the minimum and maximum number of arguments
+expected by the function (excluding the implicit 'name' argument). If
+<max.args> is None, there is no upper limit to the number of arguments. Passing
+an invalid number of arguments will generate a KconfigError exception.
+
+Functions can access the current parsing location as kconf.filename/linenr.
+Accessing other fields of the Kconfig object is not safe. See the warning
+below.
+
+Keep in mind that for a variable defined like 'foo = $(fn)', 'fn' will be
+called only when 'foo' is expanded. If 'fn' uses the parsing location and the
+intent is to use the location of the assignment, you want 'foo := $(fn)'
+instead, which calls the function immediately.
+
+Once defined, user functions can be called from Kconfig in the same way as
+other preprocessor functions:
+
+ config FOO
+ ...
+ depends on $(my-fn,arg1,arg2)
+
+If my_fn() returns "n", this will result in
+
+ config FOO
+ ...
+ depends on n
+
+Warning
+*******
+
+User-defined preprocessor functions are called as they're encountered at parse
+time, before all Kconfig files have been processed, and before the menu tree
+has been finalized. There are no guarantees that accessing Kconfig symbols or
+the menu tree via the 'kconf' parameter will work, and it could potentially
+lead to a crash.
+
+Preferably, user-defined functions should be stateless.
+
+
+Feedback
+========
+
+Send bug reports, suggestions, and questions to ulfalizer a.t Google's email
+service, or open a ticket on the GitHub page.
+"""
+import errno
+import importlib
+import os
+import re
+import sys
+
+# Get rid of some attribute lookups. These are obvious in context.
+from glob import iglob
+from os.path import dirname, exists, expandvars, islink, join, realpath
+
+
+VERSION = (14, 1, 0)
+
+
+# File layout:
+#
+# Public classes
+# Public functions
+# Internal functions
+# Global constants
+
+# Line length: 79 columns
+
+
+#
+# Public classes
+#
+
+
+class Kconfig(object):
+ """
+ Represents a Kconfig configuration, e.g. for x86 or ARM. This is the set of
+ symbols, choices, and menu nodes appearing in the configuration. Creating
+ any number of Kconfig objects (including for different architectures) is
+ safe. Kconfiglib doesn't keep any global state.
+
+ The following attributes are available. They should be treated as
+ read-only, and some are implemented through @property magic.
+
+ syms:
+ A dictionary with all symbols in the configuration, indexed by name. Also
+ includes all symbols that are referenced in expressions but never
+ defined, except for constant (quoted) symbols.
+
+ Undefined symbols can be recognized by Symbol.nodes being empty -- see
+ the 'Intro to the menu tree' section in the module docstring.
+
+ const_syms:
+ A dictionary like 'syms' for constant (quoted) symbols
+
+ named_choices:
+ A dictionary like 'syms' for named choices (choice FOO)
+
+ defined_syms:
+ A list with all defined symbols, in the same order as they appear in the
+ Kconfig files. Symbols defined in multiple locations appear multiple
+ times.
+
+ Note: You probably want to use 'unique_defined_syms' instead. This
+ attribute is mostly maintained for backwards compatibility.
+
+ unique_defined_syms:
+ A list like 'defined_syms', but with duplicates removed. Just the first
+ instance is kept for symbols defined in multiple locations. Kconfig order
+ is preserved otherwise.
+
+ Using this attribute instead of 'defined_syms' can save work, and
+ automatically gives reasonable behavior when writing configuration output
+ (symbols defined in multiple locations only generate output once, while
+ still preserving Kconfig order for readability).
+
+ choices:
+ A list with all choices, in the same order as they appear in the Kconfig
+ files.
+
+ Note: You probably want to use 'unique_choices' instead. This attribute
+ is mostly maintained for backwards compatibility.
+
+ unique_choices:
+ Analogous to 'unique_defined_syms', for choices. Named choices can have
+ multiple definition locations.
+
+ menus:
+ A list with all menus, in the same order as they appear in the Kconfig
+ files
+
+ comments:
+ A list with all comments, in the same order as they appear in the Kconfig
+ files
+
+ kconfig_filenames:
+ A list with the filenames of all Kconfig files included in the
+ configuration, relative to $srctree (or relative to the current directory
+ if $srctree isn't set), except absolute paths (e.g.
+ 'source "/foo/Kconfig"') are kept as-is.
+
+ The files are listed in the order they are source'd, starting with the
+ top-level Kconfig file. If a file is source'd multiple times, it will
+ appear multiple times. Use set() to get unique filenames.
+
+ Note that Kconfig.sync_deps() already indirectly catches any file
+ modifications that change configuration output.
+
+ env_vars:
+ A set() with the names of all environment variables referenced in the
+ Kconfig files.
+
+ Only environment variables referenced with the preprocessor $(FOO) syntax
+ will be registered. The older $FOO syntax is only supported for backwards
+ compatibility.
+
+ Also note that $(FOO) won't be registered unless the environment variable
+ $FOO is actually set. If it isn't, $(FOO) is an expansion of an unset
+ preprocessor variable (which gives the empty string).
+
+ Another gotcha is that environment variables referenced in the values of
+ recursively expanded preprocessor variables (those defined with =) will
+ only be registered if the variable is actually used (expanded) somewhere.
+
+ The note from the 'kconfig_filenames' documentation applies here too.
+
+ n/m/y:
+ The predefined constant symbols n/m/y. Also available in const_syms.
+
+ modules:
+ The Symbol instance for the modules symbol. Currently hardcoded to
+ MODULES, which is backwards compatible. Kconfiglib will warn if
+ 'option modules' is set on some other symbol. Tell me if you need proper
+ 'option modules' support.
+
+ 'modules' is never None. If the MODULES symbol is not explicitly defined,
+ its tri_value will be 0 (n), as expected.
+
+ A simple way to enable modules is to do 'kconf.modules.set_value(2)'
+ (provided the MODULES symbol is defined and visible). Modules are
+ disabled by default in the kernel Kconfig files as of writing, though
+ nearly all defconfig files enable them (with 'CONFIG_MODULES=y').
+
+ defconfig_list:
+ The Symbol instance for the 'option defconfig_list' symbol, or None if no
+ defconfig_list symbol exists. The defconfig filename derived from this
+ symbol can be found in Kconfig.defconfig_filename.
+
+ defconfig_filename:
+ The filename given by the defconfig_list symbol. This is taken from the
+ first 'default' with a satisfied condition where the specified file
+ exists (can be opened for reading). If a defconfig file foo/defconfig is
+ not found and $srctree was set when the Kconfig was created,
+ $srctree/foo/defconfig is looked up as well.
+
+ 'defconfig_filename' is None if either no defconfig_list symbol exists,
+ or if the defconfig_list symbol has no 'default' with a satisfied
+ condition that specifies a file that exists.
+
+ Gotcha: scripts/kconfig/Makefile might pass --defconfig=<defconfig> to
+ scripts/kconfig/conf when running e.g. 'make defconfig'. This option
+ overrides the defconfig_list symbol, meaning defconfig_filename might not
+ always match what 'make defconfig' would use.
+
+ top_node:
+ The menu node (see the MenuNode class) of the implicit top-level menu.
+ Acts as the root of the menu tree.
+
+ mainmenu_text:
+ The prompt (title) of the top menu (top_node). Defaults to "Main menu".
+ Can be changed with the 'mainmenu' statement (see kconfig-language.txt).
+
+ variables:
+ A dictionary with all preprocessor variables, indexed by name. See the
+ Variable class.
+
+ warn:
+ Set this variable to True/False to enable/disable warnings. See
+ Kconfig.__init__().
+
+ When 'warn' is False, the values of the other warning-related variables
+ are ignored.
+
+ This variable as well as the other warn* variables can be read to check
+ the current warning settings.
+
+ warn_to_stderr:
+ Set this variable to True/False to enable/disable warnings on stderr. See
+ Kconfig.__init__().
+
+ warn_assign_undef:
+ Set this variable to True to generate warnings for assignments to
+ undefined symbols in configuration files.
+
+ This variable is False by default unless the KCONFIG_WARN_UNDEF_ASSIGN
+ environment variable was set to 'y' when the Kconfig instance was
+ created.
+
+ warn_assign_override:
+ Set this variable to True to generate warnings for multiple assignments
+ to the same symbol in configuration files, where the assignments set
+ different values (e.g. CONFIG_FOO=m followed by CONFIG_FOO=y, where the
+ last value would get used).
+
+ This variable is True by default. Disabling it might be useful when
+ merging configurations.
+
+ warn_assign_redun:
+ Like warn_assign_override, but for multiple assignments setting a symbol
+ to the same value.
+
+ This variable is True by default. Disabling it might be useful when
+ merging configurations.
+
+ warnings:
+ A list of strings containing all warnings that have been generated, for
+ cases where more flexibility is needed.
+
+ See the 'warn_to_stderr' parameter to Kconfig.__init__() and the
+ Kconfig.warn_to_stderr variable as well. Note that warnings still get
+ added to Kconfig.warnings when 'warn_to_stderr' is True.
+
+ Just as for warnings printed to stderr, only warnings that are enabled
+ will get added to Kconfig.warnings. See the various Kconfig.warn*
+ variables.
+
+ missing_syms:
+ A list with (name, value) tuples for all assignments to undefined symbols
+ within the most recently loaded .config file(s). 'name' is the symbol
+ name without the 'CONFIG_' prefix. 'value' is a string that gives the
+ right-hand side of the assignment verbatim.
+
+ See Kconfig.load_config() as well.
+
+ srctree:
+ The value the $srctree environment variable had when the Kconfig instance
+ was created, or the empty string if $srctree wasn't set. This gives nice
+ behavior with os.path.join(), which treats "" as the current directory,
+ without adding "./".
+
+ Kconfig files are looked up relative to $srctree (unless absolute paths
+ are used), and .config files are looked up relative to $srctree if they
+ are not found in the current directory. This is used to support
+ out-of-tree builds. The C tools use this environment variable in the same
+ way.
+
+ Changing $srctree after creating the Kconfig instance has no effect. Only
+ the value when the configuration is loaded matters. This avoids surprises
+ if multiple configurations are loaded with different values for $srctree.
+
+ config_prefix:
+ The value the CONFIG_ environment variable had when the Kconfig instance
+ was created, or "CONFIG_" if CONFIG_ wasn't set. This is the prefix used
+ (and expected) on symbol names in .config files and C headers. Used in
+ the same way in the C tools.
+
+ config_header:
+ The value the KCONFIG_CONFIG_HEADER environment variable had when the
+ Kconfig instance was created, or the empty string if
+ KCONFIG_CONFIG_HEADER wasn't set. This string is inserted verbatim at the
+ beginning of configuration files. See write_config().
+
+ header_header:
+ The value the KCONFIG_AUTOHEADER_HEADER environment variable had when the
+ Kconfig instance was created, or the empty string if
+ KCONFIG_AUTOHEADER_HEADER wasn't set. This string is inserted verbatim at
+ the beginning of header files. See write_autoconf().
+
+ filename/linenr:
+ The current parsing location, for use in Python preprocessor functions.
+ See the module docstring.
+ """
+ __slots__ = (
+ "_encoding",
+ "_functions",
+ "_set_match",
+ "_srctree_prefix",
+ "_unset_match",
+ "_warn_assign_no_prompt",
+ "choices",
+ "comments",
+ "config_header",
+ "config_prefix",
+ "const_syms",
+ "defconfig_list",
+ "defined_syms",
+ "env_vars",
+ "header_header",
+ "kconfig_filenames",
+ "m",
+ "menus",
+ "missing_syms",
+ "modules",
+ "n",
+ "named_choices",
+ "srctree",
+ "syms",
+ "top_node",
+ "unique_choices",
+ "unique_defined_syms",
+ "variables",
+ "warn",
+ "warn_assign_override",
+ "warn_assign_redun",
+ "warn_assign_undef",
+ "warn_to_stderr",
+ "warnings",
+ "y",
+
+ # Parsing-related
+ "_parsing_kconfigs",
+ "_readline",
+ "filename",
+ "linenr",
+ "_include_path",
+ "_filestack",
+ "_line",
+ "_tokens",
+ "_tokens_i",
+ "_reuse_tokens",
+ )
+
+ #
+ # Public interface
+ #
+
+ def __init__(self, filename="Kconfig", warn=True, warn_to_stderr=True,
+ encoding="utf-8", suppress_traceback=False):
+ """
+ Creates a new Kconfig object by parsing Kconfig files.
+ Note that Kconfig files are not the same as .config files (which store
+ configuration symbol values).
+
+ See the module docstring for some environment variables that influence
+ default warning settings (KCONFIG_WARN_UNDEF and
+ KCONFIG_WARN_UNDEF_ASSIGN).
+
+ Raises KconfigError on syntax/semantic errors, and OSError or (possibly
+ a subclass of) IOError on IO errors ('errno', 'strerror', and
+ 'filename' are available). Note that IOError is an alias for OSError on
+ Python 3, so it's enough to catch OSError there. If you need Python 2/3
+ compatibility, it's easiest to catch EnvironmentError, which is a
+ common base class of OSError/IOError on Python 2 and an alias for
+ OSError on Python 3.
+
+ filename (default: "Kconfig"):
+ The Kconfig file to load. For the Linux kernel, you'll want "Kconfig"
+ from the top-level directory, as environment variables will make sure
+ the right Kconfig is included from there (arch/$SRCARCH/Kconfig as of
+ writing).
+
+ If $srctree is set, 'filename' will be looked up relative to it.
+ $srctree is also used to look up source'd files within Kconfig files.
+ See the class documentation.
+
+ If you are using Kconfiglib via 'make scriptconfig', the filename of
+ the base base Kconfig file will be in sys.argv[1]. It's currently
+ always "Kconfig" in practice.
+
+ warn (default: True):
+ True if warnings related to this configuration should be generated.
+ This can be changed later by setting Kconfig.warn to True/False. It
+ is provided as a constructor argument since warnings might be
+ generated during parsing.
+
+ See the other Kconfig.warn_* variables as well, which enable or
+ suppress certain warnings when warnings are enabled.
+
+ All generated warnings are added to the Kconfig.warnings list. See
+ the class documentation.
+
+ warn_to_stderr (default: True):
+ True if warnings should be printed to stderr in addition to being
+ added to Kconfig.warnings.
+
+ This can be changed later by setting Kconfig.warn_to_stderr to
+ True/False.
+
+ encoding (default: "utf-8"):
+ The encoding to use when reading and writing files, and when decoding
+ output from commands run via $(shell). If None, the encoding
+ specified in the current locale will be used.
+
+ The "utf-8" default avoids exceptions on systems that are configured
+ to use the C locale, which implies an ASCII encoding.
+
+ This parameter has no effect on Python 2, due to implementation
+ issues (regular strings turning into Unicode strings, which are
+ distinct in Python 2). Python 2 doesn't decode regular strings
+ anyway.
+
+ Related PEP: https://www.python.org/dev/peps/pep-0538/
+
+ suppress_traceback (default: False):
+ Helper for tools. When True, any EnvironmentError or KconfigError
+ generated during parsing is caught, the exception message is printed
+ to stderr together with the command name, and sys.exit(1) is called
+ (which generates SystemExit).
+
+ This hides the Python traceback for "expected" errors like syntax
+ errors in Kconfig files.
+
+ Other exceptions besides EnvironmentError and KconfigError are still
+ propagated when suppress_traceback is True.
+ """
+ try:
+ self._init(filename, warn, warn_to_stderr, encoding)
+ except (EnvironmentError, KconfigError) as e:
+ if suppress_traceback:
+ cmd = sys.argv[0] # Empty string if missing
+ if cmd:
+ cmd += ": "
+ # Some long exception messages have extra newlines for better
+ # formatting when reported as an unhandled exception. Strip
+ # them here.
+ sys.exit(cmd + str(e).strip())
+ raise
+
+ def _init(self, filename, warn, warn_to_stderr, encoding):
+ # See __init__()
+
+ self._encoding = encoding
+
+ self.srctree = os.getenv("srctree", "")
+ # A prefix we can reliably strip from glob() results to get a filename
+ # relative to $srctree. relpath() can cause issues for symlinks,
+ # because it assumes symlink/../foo is the same as foo/.
+ self._srctree_prefix = realpath(self.srctree) + os.sep
+
+ self.warn = warn
+ self.warn_to_stderr = warn_to_stderr
+ self.warn_assign_undef = os.getenv("KCONFIG_WARN_UNDEF_ASSIGN") == "y"
+ self.warn_assign_override = True
+ self.warn_assign_redun = True
+ self._warn_assign_no_prompt = True
+
+ self.warnings = []
+
+ self.config_prefix = os.getenv("CONFIG_", "CONFIG_")
+ # Regular expressions for parsing .config files
+ self._set_match = _re_match(self.config_prefix + r"([^=]+)=(.*)")
+ self._unset_match = _re_match(r"# {}([^ ]+) is not set".format(
+ self.config_prefix))
+
+ self.config_header = os.getenv("KCONFIG_CONFIG_HEADER", "")
+ self.header_header = os.getenv("KCONFIG_AUTOHEADER_HEADER", "")
+
+ self.syms = {}
+ self.const_syms = {}
+ self.defined_syms = []
+ self.missing_syms = []
+ self.named_choices = {}
+ self.choices = []
+ self.menus = []
+ self.comments = []
+
+ for nmy in "n", "m", "y":
+ sym = Symbol()
+ sym.kconfig = self
+ sym.name = nmy
+ sym.is_constant = True
+ sym.orig_type = TRISTATE
+ sym._cached_tri_val = STR_TO_TRI[nmy]
+
+ self.const_syms[nmy] = sym
+
+ self.n = self.const_syms["n"]
+ self.m = self.const_syms["m"]
+ self.y = self.const_syms["y"]
+
+ # Make n/m/y well-formed symbols
+ for nmy in "n", "m", "y":
+ sym = self.const_syms[nmy]
+ sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
+
+ # Maps preprocessor variables names to Variable instances
+ self.variables = {}
+
+ # Predefined preprocessor functions, with min/max number of arguments
+ self._functions = {
+ "info": (_info_fn, 1, 1),
+ "error-if": (_error_if_fn, 2, 2),
+ "filename": (_filename_fn, 0, 0),
+ "lineno": (_lineno_fn, 0, 0),
+ "shell": (_shell_fn, 1, 1),
+ "warning-if": (_warning_if_fn, 2, 2),
+ }
+
+ # Add any user-defined preprocessor functions
+ try:
+ self._functions.update(
+ importlib.import_module(
+ os.getenv("KCONFIG_FUNCTIONS", "kconfigfunctions")
+ ).functions)
+ except ImportError:
+ pass
+
+ # This determines whether previously unseen symbols are registered.
+ # They shouldn't be if we parse expressions after parsing, as part of
+ # Kconfig.eval_string().
+ self._parsing_kconfigs = True
+
+ self.modules = self._lookup_sym("MODULES")
+ self.defconfig_list = None
+
+ self.top_node = MenuNode()
+ self.top_node.kconfig = self
+ self.top_node.item = MENU
+ self.top_node.is_menuconfig = True
+ self.top_node.visibility = self.y
+ self.top_node.prompt = ("Main menu", self.y)
+ self.top_node.parent = None
+ self.top_node.dep = self.y
+ self.top_node.filename = filename
+ self.top_node.linenr = 1
+ self.top_node.include_path = ()
+
+ # Parse the Kconfig files
+
+ # Not used internally. Provided as a convenience.
+ self.kconfig_filenames = [filename]
+ self.env_vars = set()
+
+ # Keeps track of the location in the parent Kconfig files. Kconfig
+ # files usually source other Kconfig files. See _enter_file().
+ self._filestack = []
+ self._include_path = ()
+
+ # The current parsing location
+ self.filename = filename
+ self.linenr = 0
+
+ # Used to avoid retokenizing lines when we discover that they're not
+ # part of the construct currently being parsed. This is kinda like an
+ # unget operation.
+ self._reuse_tokens = False
+
+ # Open the top-level Kconfig file. Store the readline() method directly
+ # as a small optimization.
+ self._readline = self._open(join(self.srctree, filename), "r").readline
+
+ try:
+ # Parse the Kconfig files. Returns the last node, which we
+ # terminate with '.next = None'.
+ self._parse_block(None, self.top_node, self.top_node).next = None
+ self.top_node.list = self.top_node.next
+ self.top_node.next = None
+ except UnicodeDecodeError as e:
+ _decoding_error(e, self.filename)
+
+ # Close the top-level Kconfig file. __self__ fetches the 'file' object
+ # for the method.
+ self._readline.__self__.close()
+
+ self._parsing_kconfigs = False
+
+ # Do various menu tree post-processing
+ self._finalize_node(self.top_node, self.y)
+
+ self.unique_defined_syms = _ordered_unique(self.defined_syms)
+ self.unique_choices = _ordered_unique(self.choices)
+
+ # Do sanity checks. Some of these depend on everything being finalized.
+ self._check_sym_sanity()
+ self._check_choice_sanity()
+
+ # KCONFIG_STRICT is an older alias for KCONFIG_WARN_UNDEF, supported
+ # for backwards compatibility
+ if os.getenv("KCONFIG_WARN_UNDEF") == "y" or \
+ os.getenv("KCONFIG_STRICT") == "y":
+
+ self._check_undef_syms()
+
+ # Build Symbol._dependents for all symbols and choices
+ self._build_dep()
+
+ # Check for dependency loops
+ check_dep_loop_sym = _check_dep_loop_sym # Micro-optimization
+ for sym in self.unique_defined_syms:
+ check_dep_loop_sym(sym, False)
+
+ # Add extra dependencies from choices to choice symbols that get
+ # awkward during dependency loop detection
+ self._add_choice_deps()
+
+ @property
+ def mainmenu_text(self):
+ """
+ See the class documentation.
+ """
+ return self.top_node.prompt[0]
+
+ @property
+ def defconfig_filename(self):
+ """
+ See the class documentation.
+ """
+ if self.defconfig_list:
+ for filename, cond in self.defconfig_list.defaults:
+ if expr_value(cond):
+ try:
+ with self._open_config(filename.str_value) as f:
+ return f.name
+ except EnvironmentError:
+ continue
+
+ return None
+
+ def load_config(self, filename=None, replace=True, verbose=None):
+ """
+ Loads symbol values from a file in the .config format. Equivalent to
+ calling Symbol.set_value() to set each of the values.
+
+ "# CONFIG_FOO is not set" within a .config file sets the user value of
+ FOO to n. The C tools work the same way.
+
+ For each symbol, the Symbol.user_value attribute holds the value the
+ symbol was assigned in the .config file (if any). The user value might
+ differ from Symbol.str/tri_value if there are unsatisfied dependencies.
+
+ Calling this function also updates the Kconfig.missing_syms attribute
+ with a list of all assignments to undefined symbols within the
+ configuration file. Kconfig.missing_syms is cleared if 'replace' is
+ True, and appended to otherwise. See the documentation for
+ Kconfig.missing_syms as well.
+
+ See the Kconfig.__init__() docstring for raised exceptions
+ (OSError/IOError). KconfigError is never raised here.
+
+ filename (default: None):
+ Path to load configuration from (a string). Respects $srctree if set
+ (see the class documentation).
+
+ If 'filename' is None (the default), the configuration file to load
+ (if any) is calculated automatically, giving the behavior you'd
+ usually want:
+
+ 1. If the KCONFIG_CONFIG environment variable is set, it gives the
+ path to the configuration file to load. Otherwise, ".config" is
+ used. See standard_config_filename().
+
+ 2. If the path from (1.) doesn't exist, the configuration file
+ given by kconf.defconfig_filename is loaded instead, which is
+ derived from the 'option defconfig_list' symbol.
+
+ 3. If (1.) and (2.) fail to find a configuration file to load, no
+ configuration file is loaded, and symbols retain their current
+ values (e.g., their default values). This is not an error.
+
+ See the return value as well.
+
+ replace (default: True):
+ If True, all existing user values will be cleared before loading the
+ .config. Pass False to merge configurations.
+
+ verbose (default: None):
+ Limited backwards compatibility to prevent crashes. A warning is
+ printed if anything but None is passed.
+
+ Prior to Kconfiglib 12.0.0, this option enabled printing of messages
+ to stdout when 'filename' was None. A message is (always) returned
+ now instead, which is more flexible.
+
+ Will probably be removed in some future version.
+
+ Returns a string with a message saying which file got loaded (or
+ possibly that no file got loaded, when 'filename' is None). This is
+ meant to reduce boilerplate in tools, which can do e.g.
+ print(kconf.load_config()). The returned message distinguishes between
+ loading (replace == True) and merging (replace == False).
+ """
+ if verbose is not None:
+ _warn_verbose_deprecated("load_config")
+
+ msg = None
+ if filename is None:
+ filename = standard_config_filename()
+ if not exists(filename) and \
+ not exists(join(self.srctree, filename)):
+ defconfig = self.defconfig_filename
+ if defconfig is None:
+ return "Using default symbol values (no '{}')" \
+ .format(filename)
+
+ msg = " default configuration '{}' (no '{}')" \
+ .format(defconfig, filename)
+ filename = defconfig
+
+ if not msg:
+ msg = " configuration '{}'".format(filename)
+
+ # Disable the warning about assigning to symbols without prompts. This
+ # is normal and expected within a .config file.
+ self._warn_assign_no_prompt = False
+
+ # This stub only exists to make sure _warn_assign_no_prompt gets
+ # reenabled
+ try:
+ self._load_config(filename, replace)
+ except UnicodeDecodeError as e:
+ _decoding_error(e, filename)
+ finally:
+ self._warn_assign_no_prompt = True
+
+ return ("Loaded" if replace else "Merged") + msg
+
+ def _load_config(self, filename, replace):
+ with self._open_config(filename) as f:
+ if replace:
+ self.missing_syms = []
+
+ # If we're replacing the configuration, keep track of which
+ # symbols and choices got set so that we can unset the rest
+ # later. This avoids invalidating everything and is faster.
+ # Another benefit is that invalidation must be rock solid for
+ # it to work, making it a good test.
+
+ for sym in self.unique_defined_syms:
+ sym._was_set = False
+
+ for choice in self.unique_choices:
+ choice._was_set = False
+
+ # Small optimizations
+ set_match = self._set_match
+ unset_match = self._unset_match
+ get_sym = self.syms.get
+
+ for linenr, line in enumerate(f, 1):
+ # The C tools ignore trailing whitespace
+ line = line.rstrip()
+
+ match = set_match(line)
+ if match:
+ name, val = match.groups()
+ sym = get_sym(name)
+ if not sym or not sym.nodes:
+ self._undef_assign(name, val, filename, linenr)
+ continue
+
+ if sym.orig_type in _BOOL_TRISTATE:
+ # The C implementation only checks the first character
+ # to the right of '=', for whatever reason
+ if not (sym.orig_type is BOOL
+ and val.startswith(("y", "n")) or
+ sym.orig_type is TRISTATE
+ and val.startswith(("y", "m", "n"))):
+ self._warn("'{}' is not a valid value for the {} "
+ "symbol {}. Assignment ignored."
+ .format(val, TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc),
+ filename, linenr)
+ continue
+
+ val = val[0]
+
+ if sym.choice and val != "n":
+ # During .config loading, we infer the mode of the
+ # choice from the kind of values that are assigned
+ # to the choice symbols
+
+ prev_mode = sym.choice.user_value
+ if prev_mode is not None and \
+ TRI_TO_STR[prev_mode] != val:
+
+ self._warn("both m and y assigned to symbols "
+ "within the same choice",
+ filename, linenr)
+
+ # Set the choice's mode
+ sym.choice.set_value(val)
+
+ elif sym.orig_type is STRING:
+ match = _conf_string_match(val)
+ if not match:
+ self._warn("malformed string literal in "
+ "assignment to {}. Assignment ignored."
+ .format(sym.name_and_loc),
+ filename, linenr)
+ continue
+
+ val = unescape(match.group(1))
+
+ else:
+ match = unset_match(line)
+ if not match:
+ # Print a warning for lines that match neither
+ # set_match() nor unset_match() and that are not blank
+ # lines or comments. 'line' has already been
+ # rstrip()'d, so blank lines show up as "" here.
+ if line and not line.lstrip().startswith("#"):
+ self._warn("ignoring malformed line '{}'"
+ .format(line),
+ filename, linenr)
+
+ continue
+
+ name = match.group(1)
+ sym = get_sym(name)
+ if not sym or not sym.nodes:
+ self._undef_assign(name, "n", filename, linenr)
+ continue
+
+ if sym.orig_type not in _BOOL_TRISTATE:
+ continue
+
+ val = "n"
+
+ # Done parsing the assignment. Set the value.
+
+ if sym._was_set:
+ self._assigned_twice(sym, val, filename, linenr)
+
+ sym.set_value(val)
+
+ if replace:
+ # If we're replacing the configuration, unset the symbols that
+ # didn't get set
+
+ for sym in self.unique_defined_syms:
+ if not sym._was_set:
+ sym.unset_value()
+
+ for choice in self.unique_choices:
+ if not choice._was_set:
+ choice.unset_value()
+
+ def _undef_assign(self, name, val, filename, linenr):
+ # Called for assignments to undefined symbols during .config loading
+
+ self.missing_syms.append((name, val))
+ if self.warn_assign_undef:
+ self._warn(
+ "attempt to assign the value '{}' to the undefined symbol {}"
+ .format(val, name), filename, linenr)
+
+ def _assigned_twice(self, sym, new_val, filename, linenr):
+ # Called when a symbol is assigned more than once in a .config file
+
+ # Use strings for bool/tristate user values in the warning
+ if sym.orig_type in _BOOL_TRISTATE:
+ user_val = TRI_TO_STR[sym.user_value]
+ else:
+ user_val = sym.user_value
+
+ msg = '{} set more than once. Old value "{}", new value "{}".'.format(
+ sym.name_and_loc, user_val, new_val)
+
+ if user_val == new_val:
+ if self.warn_assign_redun:
+ self._warn(msg, filename, linenr)
+ elif self.warn_assign_override:
+ self._warn(msg, filename, linenr)
+
+ def load_allconfig(self, filename):
+ """
+ Helper for all*config. Loads (merges) the configuration file specified
+ by KCONFIG_ALLCONFIG, if any. See Documentation/kbuild/kconfig.txt in
+ the Linux kernel.
+
+ Disables warnings for duplicated assignments within configuration files
+ for the duration of the call
+ (kconf.warn_assign_override/warn_assign_redun = False), and restores
+ the previous warning settings at the end. The KCONFIG_ALLCONFIG
+ configuration file is expected to override symbols.
+
+ Exits with sys.exit() (which raises a SystemExit exception) and prints
+ an error to stderr if KCONFIG_ALLCONFIG is set but the configuration
+ file can't be opened.
+
+ filename:
+ Command-specific configuration filename - "allyes.config",
+ "allno.config", etc.
+ """
+ load_allconfig(self, filename)
+
+ def write_autoconf(self, filename=None, header=None):
+ r"""
+ Writes out symbol values as a C header file, matching the format used
+ by include/generated/autoconf.h in the kernel.
+
+ The ordering of the #defines matches the one generated by
+ write_config(). The order in the C implementation depends on the hash
+ table implementation as of writing, and so won't match.
+
+ If 'filename' exists and its contents is identical to what would get
+ written out, it is left untouched. This avoids updating file metadata
+ like the modification time and possibly triggering redundant work in
+ build tools.
+
+ filename (default: None):
+ Path to write header to.
+
+ If None (the default), the path in the environment variable
+ KCONFIG_AUTOHEADER is used if set, and "include/generated/autoconf.h"
+ otherwise. This is compatible with the C tools.
+
+ header (default: None):
+ Text inserted verbatim at the beginning of the file. You would
+ usually want it enclosed in '/* */' to make it a C comment, and
+ include a trailing newline.
+
+ If None (the default), the value of the environment variable
+ KCONFIG_AUTOHEADER_HEADER had when the Kconfig instance was created
+ will be used if it was set, and no header otherwise. See the
+ Kconfig.header_header attribute.
+
+ Returns a string with a message saying that the header got saved, or
+ that there were no changes to it. This is meant to reduce boilerplate
+ in tools, which can do e.g. print(kconf.write_autoconf()).
+ """
+ if filename is None:
+ filename = os.getenv("KCONFIG_AUTOHEADER",
+ "include/generated/autoconf.h")
+
+ if self._write_if_changed(filename, self._autoconf_contents(header)):
+ return "Kconfig header saved to '{}'".format(filename)
+ return "No change to Kconfig header in '{}'".format(filename)
+
+ def _autoconf_contents(self, header):
+ # write_autoconf() helper. Returns the contents to write as a string,
+ # with 'header' or KCONFIG_AUTOHEADER_HEADER at the beginning.
+
+ if header is None:
+ header = self.header_header
+
+ chunks = [header] # "".join()ed later
+ add = chunks.append
+
+ for sym in self.unique_defined_syms:
+ # _write_to_conf is determined when the value is calculated. This
+ # is a hidden function call due to property magic.
+ #
+ # Note: In client code, you can check if sym.config_string is empty
+ # instead, to avoid accessing the internal _write_to_conf variable
+ # (though it's likely to keep working).
+ val = sym.str_value
+ if not sym._write_to_conf:
+ continue
+
+ if sym.orig_type in _BOOL_TRISTATE:
+ if val == "y":
+ add("#define {}{} 1\n"
+ .format(self.config_prefix, sym.name))
+ elif val == "m":
+ add("#define {}{}_MODULE 1\n"
+ .format(self.config_prefix, sym.name))
+
+ elif sym.orig_type is STRING:
+ add('#define {}{} "{}"\n'
+ .format(self.config_prefix, sym.name, escape(val)))
+
+ else: # sym.orig_type in _INT_HEX:
+ if sym.orig_type is HEX and \
+ not val.startswith(("0x", "0X")):
+ val = "0x" + val
+
+ add("#define {}{} {}\n"
+ .format(self.config_prefix, sym.name, val))
+
+ return "".join(chunks)
+
+ def write_config(self, filename=None, header=None, save_old=True,
+ verbose=None):
+ r"""
+ Writes out symbol values in the .config format. The format matches the
+ C implementation, including ordering.
+
+ Symbols appear in the same order in generated .config files as they do
+ in the Kconfig files. For symbols defined in multiple locations, a
+ single assignment is written out corresponding to the first location
+ where the symbol is defined.
+
+ See the 'Intro to symbol values' section in the module docstring to
+ understand which symbols get written out.
+
+ If 'filename' exists and its contents is identical to what would get
+ written out, it is left untouched. This avoids updating file metadata
+ like the modification time and possibly triggering redundant work in
+ build tools.
+
+ See the Kconfig.__init__() docstring for raised exceptions
+ (OSError/IOError). KconfigError is never raised here.
+
+ filename (default: None):
+ Path to write configuration to (a string).
+
+ If None (the default), the path in the environment variable
+ KCONFIG_CONFIG is used if set, and ".config" otherwise. See
+ standard_config_filename().
+
+ header (default: None):
+ Text inserted verbatim at the beginning of the file. You would
+ usually want each line to start with '#' to make it a comment, and
+ include a trailing newline.
+
+ if None (the default), the value of the environment variable
+ KCONFIG_CONFIG_HEADER had when the Kconfig instance was created will
+ be used if it was set, and no header otherwise. See the
+ Kconfig.config_header attribute.
+
+ save_old (default: True):
+ If True and <filename> already exists, a copy of it will be saved to
+ <filename>.old in the same directory before the new configuration is
+ written.
+
+ Errors are silently ignored if <filename>.old cannot be written (e.g.
+ due to being a directory, or <filename> being something like
+ /dev/null).
+
+ verbose (default: None):
+ Limited backwards compatibility to prevent crashes. A warning is
+ printed if anything but None is passed.
+
+ Prior to Kconfiglib 12.0.0, this option enabled printing of messages
+ to stdout when 'filename' was None. A message is (always) returned
+ now instead, which is more flexible.
+
+ Will probably be removed in some future version.
+
+ Returns a string with a message saying which file got saved. This is
+ meant to reduce boilerplate in tools, which can do e.g.
+ print(kconf.write_config()).
+ """
+ if verbose is not None:
+ _warn_verbose_deprecated("write_config")
+
+ if filename is None:
+ filename = standard_config_filename()
+
+ contents = self._config_contents(header)
+ if self._contents_eq(filename, contents):
+ return "No change to configuration in '{}'".format(filename)
+
+ if save_old:
+ _save_old(filename)
+
+ with self._open(filename, "w") as f:
+ f.write(contents)
+
+ return "Configuration saved to '{}'".format(filename)
+
+ def _config_contents(self, header):
+ # write_config() helper. Returns the contents to write as a string,
+ # with 'header' or KCONFIG_CONFIG_HEADER at the beginning.
+ #
+ # More memory friendly would be to 'yield' the strings and
+ # "".join(_config_contents()), but it was a bit slower on my system.
+
+ # node_iter() was used here before commit 3aea9f7 ("Add '# end of
+ # <menu>' after menus in .config"). Those comments get tricky to
+ # implement with it.
+
+ for sym in self.unique_defined_syms:
+ sym._visited = False
+
+ if header is None:
+ header = self.config_header
+
+ chunks = [header] # "".join()ed later
+ add = chunks.append
+
+ # Did we just print an '# end of ...' comment?
+ after_end_comment = False
+
+ node = self.top_node
+ while 1:
+ # Jump to the next node with an iterative tree walk
+ if node.list:
+ node = node.list
+ elif node.next:
+ node = node.next
+ else:
+ while node.parent:
+ node = node.parent
+
+ # Add a comment when leaving visible menus
+ if node.item is MENU and expr_value(node.dep) and \
+ expr_value(node.visibility) and \
+ node is not self.top_node:
+ add("# end of {}\n".format(node.prompt[0]))
+ after_end_comment = True
+
+ if node.next:
+ node = node.next
+ break
+ else:
+ # No more nodes
+ return "".join(chunks)
+
+ # Generate configuration output for the node
+
+ item = node.item
+
+ if item.__class__ is Symbol:
+ if item._visited:
+ continue
+ item._visited = True
+
+ conf_string = item.config_string
+ if not conf_string:
+ continue
+
+ if after_end_comment:
+ # Add a blank line before the first symbol printed after an
+ # '# end of ...' comment
+ after_end_comment = False
+ add("\n")
+ add(conf_string)
+
+ elif expr_value(node.dep) and \
+ ((item is MENU and expr_value(node.visibility)) or
+ item is COMMENT):
+
+ add("\n#\n# {}\n#\n".format(node.prompt[0]))
+ after_end_comment = False
+
+ def write_min_config(self, filename, header=None):
+ """
+ Writes out a "minimal" configuration file, omitting symbols whose value
+ matches their default value. The format matches the one produced by
+ 'make savedefconfig'.
+
+ The resulting configuration file is incomplete, but a complete
+ configuration can be derived from it by loading it. Minimal
+ configuration files can serve as a more manageable configuration format
+ compared to a "full" .config file, especially when configurations files
+ are merged or edited by hand.
+
+ See the Kconfig.__init__() docstring for raised exceptions
+ (OSError/IOError). KconfigError is never raised here.
+
+ filename:
+ Path to write minimal configuration to.
+
+ header (default: None):
+ Text inserted verbatim at the beginning of the file. You would
+ usually want each line to start with '#' to make it a comment, and
+ include a final terminating newline.
+
+ if None (the default), the value of the environment variable
+ KCONFIG_CONFIG_HEADER had when the Kconfig instance was created will
+ be used if it was set, and no header otherwise. See the
+ Kconfig.config_header attribute.
+
+ Returns a string with a message saying the minimal configuration got
+ saved, or that there were no changes to it. This is meant to reduce
+ boilerplate in tools, which can do e.g.
+ print(kconf.write_min_config()).
+ """
+ if self._write_if_changed(filename, self._min_config_contents(header)):
+ return "Minimal configuration saved to '{}'".format(filename)
+ return "No change to minimal configuration in '{}'".format(filename)
+
+ def _min_config_contents(self, header):
+ # write_min_config() helper. Returns the contents to write as a string,
+ # with 'header' or KCONFIG_CONFIG_HEADER at the beginning.
+
+ if header is None:
+ header = self.config_header
+
+ chunks = [header] # "".join()ed later
+ add = chunks.append
+
+ for sym in self.unique_defined_syms:
+ # Skip symbols that cannot be changed. Only check
+ # non-choice symbols, as selects don't affect choice
+ # symbols.
+ if not sym.choice and \
+ sym.visibility <= expr_value(sym.rev_dep):
+ continue
+
+ # Skip symbols whose value matches their default
+ if sym.str_value == sym._str_default():
+ continue
+
+ # Skip symbols that would be selected by default in a
+ # choice, unless the choice is optional or the symbol type
+ # isn't bool (it might be possible to set the choice mode
+ # to n or the symbol to m in those cases).
+ if sym.choice and \
+ not sym.choice.is_optional and \
+ sym.choice._selection_from_defaults() is sym and \
+ sym.orig_type is BOOL and \
+ sym.tri_value == 2:
+ continue
+
+ add(sym.config_string)
+
+ return "".join(chunks)
+
+ def sync_deps(self, path):
+ """
+ Creates or updates a directory structure that can be used to avoid
+ doing a full rebuild whenever the configuration is changed, mirroring
+ include/config/ in the kernel.
+
+ This function is intended to be called during each build, before
+ compiling source files that depend on configuration symbols.
+
+ See the Kconfig.__init__() docstring for raised exceptions
+ (OSError/IOError). KconfigError is never raised here.
+
+ path:
+ Path to directory
+
+ sync_deps(path) does the following:
+
+ 1. If the directory <path> does not exist, it is created.
+
+ 2. If <path>/auto.conf exists, old symbol values are loaded from it,
+ which are then compared against the current symbol values. If a
+ symbol has changed value (would generate different output in
+ autoconf.h compared to before), the change is signaled by
+ touch'ing a file corresponding to the symbol.
+
+ The first time sync_deps() is run on a directory, <path>/auto.conf
+ won't exist, and no old symbol values will be available. This
+ logically has the same effect as updating the entire
+ configuration.
+
+ The path to a symbol's file is calculated from the symbol's name
+ by replacing all '_' with '/' and appending '.h'. For example, the
+ symbol FOO_BAR_BAZ gets the file <path>/foo/bar/baz.h, and FOO
+ gets the file <path>/foo.h.
+
+ This scheme matches the C tools. The point is to avoid having a
+ single directory with a huge number of files, which the underlying
+ filesystem might not handle well.
+
+ 3. A new auto.conf with the current symbol values is written, to keep
+ track of them for the next build.
+
+ If auto.conf exists and its contents is identical to what would
+ get written out, it is left untouched. This avoids updating file
+ metadata like the modification time and possibly triggering
+ redundant work in build tools.
+
+
+ The last piece of the puzzle is knowing what symbols each source file
+ depends on. Knowing that, dependencies can be added from source files
+ to the files corresponding to the symbols they depends on. The source
+ file will then get recompiled (only) when the symbol value changes
+ (provided sync_deps() is run first during each build).
+
+ The tool in the kernel that extracts symbol dependencies from source
+ files is scripts/basic/fixdep.c. Missing symbol files also correspond
+ to "not changed", which fixdep deals with by using the $(wildcard) Make
+ function when adding symbol prerequisites to source files.
+
+ In case you need a different scheme for your project, the sync_deps()
+ implementation can be used as a template.
+ """
+ if not exists(path):
+ os.mkdir(path, 0o755)
+
+ # Load old values from auto.conf, if any
+ self._load_old_vals(path)
+
+ for sym in self.unique_defined_syms:
+ # _write_to_conf is determined when the value is calculated. This
+ # is a hidden function call due to property magic.
+ #
+ # Note: In client code, you can check if sym.config_string is empty
+ # instead, to avoid accessing the internal _write_to_conf variable
+ # (though it's likely to keep working).
+ val = sym.str_value
+
+ # n tristate values do not get written to auto.conf and autoconf.h,
+ # making a missing symbol logically equivalent to n
+
+ if sym._write_to_conf:
+ if sym._old_val is None and \
+ sym.orig_type in _BOOL_TRISTATE and \
+ val == "n":
+ # No old value (the symbol was missing or n), new value n.
+ # No change.
+ continue
+
+ if val == sym._old_val:
+ # New value matches old. No change.
+ continue
+
+ elif sym._old_val is None:
+ # The symbol wouldn't appear in autoconf.h (because
+ # _write_to_conf is false), and it wouldn't have appeared in
+ # autoconf.h previously either (because it didn't appear in
+ # auto.conf). No change.
+ continue
+
+ # 'sym' has a new value. Flag it.
+ _touch_dep_file(path, sym.name)
+
+ # Remember the current values as the "new old" values.
+ #
+ # This call could go anywhere after the call to _load_old_vals(), but
+ # putting it last means _sync_deps() can be safely rerun if it fails
+ # before this point.
+ self._write_old_vals(path)
+
+ def _load_old_vals(self, path):
+ # Loads old symbol values from auto.conf into a dedicated
+ # Symbol._old_val field. Mirrors load_config().
+ #
+ # The extra field could be avoided with some trickery involving dumping
+ # symbol values and restoring them later, but this is simpler and
+ # faster. The C tools also use a dedicated field for this purpose.
+
+ for sym in self.unique_defined_syms:
+ sym._old_val = None
+
+ try:
+ auto_conf = self._open(join(path, "auto.conf"), "r")
+ except EnvironmentError as e:
+ if e.errno == errno.ENOENT:
+ # No old values
+ return
+ raise
+
+ with auto_conf as f:
+ for line in f:
+ match = self._set_match(line)
+ if not match:
+ # We only expect CONFIG_FOO=... (and possibly a header
+ # comment) in auto.conf
+ continue
+
+ name, val = match.groups()
+ if name in self.syms:
+ sym = self.syms[name]
+
+ if sym.orig_type is STRING:
+ match = _conf_string_match(val)
+ if not match:
+ continue
+ val = unescape(match.group(1))
+
+ self.syms[name]._old_val = val
+ else:
+ # Flag that the symbol no longer exists, in
+ # case something still depends on it
+ _touch_dep_file(path, name)
+
+ def _write_old_vals(self, path):
+ # Helper for writing auto.conf. Basically just a simplified
+ # write_config() that doesn't write any comments (including
+ # '# CONFIG_FOO is not set' comments). The format matches the C
+ # implementation, though the ordering is arbitrary there (depends on
+ # the hash table implementation).
+ #
+ # A separate helper function is neater than complicating write_config()
+ # by passing a flag to it, plus we only need to look at symbols here.
+
+ self._write_if_changed(
+ os.path.join(path, "auto.conf"),
+ self._old_vals_contents())
+
+ def _old_vals_contents(self):
+ # _write_old_vals() helper. Returns the contents to write as a string.
+
+ # Temporary list instead of generator makes this a bit faster
+ return "".join([
+ sym.config_string for sym in self.unique_defined_syms
+ if not (sym.orig_type in _BOOL_TRISTATE and not sym.tri_value)
+ ])
+
+ def node_iter(self, unique_syms=False):
+ """
+ Returns a generator for iterating through all MenuNode's in the Kconfig
+ tree. The iteration is done in Kconfig definition order (each node is
+ visited before its children, and the children of a node are visited
+ before the next node).
+
+ The Kconfig.top_node menu node is skipped. It contains an implicit menu
+ that holds the top-level items.
+
+ As an example, the following code will produce a list equal to
+ Kconfig.defined_syms:
+
+ defined_syms = [node.item for node in kconf.node_iter()
+ if isinstance(node.item, Symbol)]
+
+ unique_syms (default: False):
+ If True, only the first MenuNode will be included for symbols defined
+ in multiple locations.
+
+ Using kconf.node_iter(True) in the example above would give a list
+ equal to unique_defined_syms.
+ """
+ if unique_syms:
+ for sym in self.unique_defined_syms:
+ sym._visited = False
+
+ node = self.top_node
+ while 1:
+ # Jump to the next node with an iterative tree walk
+ if node.list:
+ node = node.list
+ elif node.next:
+ node = node.next
+ else:
+ while node.parent:
+ node = node.parent
+ if node.next:
+ node = node.next
+ break
+ else:
+ # No more nodes
+ return
+
+ if unique_syms and node.item.__class__ is Symbol:
+ if node.item._visited:
+ continue
+ node.item._visited = True
+
+ yield node
+
+ def eval_string(self, s):
+ """
+ Returns the tristate value of the expression 's', represented as 0, 1,
+ and 2 for n, m, and y, respectively. Raises KconfigError on syntax
+ errors. Warns if undefined symbols are referenced.
+
+ As an example, if FOO and BAR are tristate symbols at least one of
+ which has the value y, then eval_string("y && (FOO || BAR)") returns
+ 2 (y).
+
+ To get the string value of non-bool/tristate symbols, use
+ Symbol.str_value. eval_string() always returns a tristate value, and
+ all non-bool/tristate symbols have the tristate value 0 (n).
+
+ The expression parsing is consistent with how parsing works for
+ conditional ('if ...') expressions in the configuration, and matches
+ the C implementation. m is rewritten to 'm && MODULES', so
+ eval_string("m") will return 0 (n) unless modules are enabled.
+ """
+ # The parser is optimized to be fast when parsing Kconfig files (where
+ # an expression can never appear at the beginning of a line). We have
+ # to monkey-patch things a bit here to reuse it.
+
+ self.filename = None
+
+ self._tokens = self._tokenize("if " + s)
+ # Strip "if " to avoid giving confusing error messages
+ self._line = s
+ self._tokens_i = 1 # Skip the 'if' token
+
+ return expr_value(self._expect_expr_and_eol())
+
+ def unset_values(self):
+ """
+ Removes any user values from all symbols, as if Kconfig.load_config()
+ or Symbol.set_value() had never been called.
+ """
+ self._warn_assign_no_prompt = False
+ try:
+ # set_value() already rejects undefined symbols, and they don't
+ # need to be invalidated (because their value never changes), so we
+ # can just iterate over defined symbols
+ for sym in self.unique_defined_syms:
+ sym.unset_value()
+
+ for choice in self.unique_choices:
+ choice.unset_value()
+ finally:
+ self._warn_assign_no_prompt = True
+
+ def enable_warnings(self):
+ """
+ Do 'Kconfig.warn = True' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn = True
+
+ def disable_warnings(self):
+ """
+ Do 'Kconfig.warn = False' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn = False
+
+ def enable_stderr_warnings(self):
+ """
+ Do 'Kconfig.warn_to_stderr = True' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn_to_stderr = True
+
+ def disable_stderr_warnings(self):
+ """
+ Do 'Kconfig.warn_to_stderr = False' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn_to_stderr = False
+
+ def enable_undef_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_undef = True' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn_assign_undef = True
+
+ def disable_undef_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_undef = False' instead. Maintained for
+ backwards compatibility.
+ """
+ self.warn_assign_undef = False
+
+ def enable_override_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_override = True' instead. Maintained for
+ backwards compatibility.
+ """
+ self.warn_assign_override = True
+
+ def disable_override_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_override = False' instead. Maintained for
+ backwards compatibility.
+ """
+ self.warn_assign_override = False
+
+ def enable_redun_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_redun = True' instead. Maintained for backwards
+ compatibility.
+ """
+ self.warn_assign_redun = True
+
+ def disable_redun_warnings(self):
+ """
+ Do 'Kconfig.warn_assign_redun = False' instead. Maintained for
+ backwards compatibility.
+ """
+ self.warn_assign_redun = False
+
+ def __repr__(self):
+ """
+ Returns a string with information about the Kconfig object when it is
+ evaluated on e.g. the interactive Python prompt.
+ """
+ def status(flag):
+ return "enabled" if flag else "disabled"
+
+ return "<{}>".format(", ".join((
+ "configuration with {} symbols".format(len(self.syms)),
+ 'main menu prompt "{}"'.format(self.mainmenu_text),
+ "srctree is current directory" if not self.srctree else
+ 'srctree "{}"'.format(self.srctree),
+ 'config symbol prefix "{}"'.format(self.config_prefix),
+ "warnings " + status(self.warn),
+ "printing of warnings to stderr " + status(self.warn_to_stderr),
+ "undef. symbol assignment warnings " +
+ status(self.warn_assign_undef),
+ "overriding symbol assignment warnings " +
+ status(self.warn_assign_override),
+ "redundant symbol assignment warnings " +
+ status(self.warn_assign_redun)
+ )))
+
+ #
+ # Private methods
+ #
+
+
+ #
+ # File reading
+ #
+
+ def _open_config(self, filename):
+ # Opens a .config file. First tries to open 'filename', then
+ # '$srctree/filename' if $srctree was set when the configuration was
+ # loaded.
+
+ try:
+ return self._open(filename, "r")
+ except EnvironmentError as e:
+ # This will try opening the same file twice if $srctree is unset,
+ # but it's not a big deal
+ try:
+ return self._open(join(self.srctree, filename), "r")
+ except EnvironmentError as e2:
+ # This is needed for Python 3, because e2 is deleted after
+ # the try block:
+ #
+ # https://docs.python.org/3/reference/compound_stmts.html#the-try-statement
+ e = e2
+
+ raise _KconfigIOError(
+ e, "Could not open '{}' ({}: {}). Check that the $srctree "
+ "environment variable ({}) is set correctly."
+ .format(filename, errno.errorcode[e.errno], e.strerror,
+ "set to '{}'".format(self.srctree) if self.srctree
+ else "unset or blank"))
+
+ def _enter_file(self, filename):
+ # Jumps to the beginning of a sourced Kconfig file, saving the previous
+ # position and file object.
+ #
+ # filename:
+ # Absolute path to file
+
+ # Path relative to $srctree, stored in e.g. self.filename (which makes
+ # it indirectly show up in MenuNode.filename). Equals 'filename' for
+ # absolute paths passed to 'source'.
+ if filename.startswith(self._srctree_prefix):
+ # Relative path (or a redundant absolute path to within $srctree,
+ # but it's probably fine to reduce those too)
+ rel_filename = filename[len(self._srctree_prefix):]
+ else:
+ # Absolute path
+ rel_filename = filename
+
+ self.kconfig_filenames.append(rel_filename)
+
+ # The parent Kconfig files are represented as a list of
+ # (<include path>, <Python 'file' object for Kconfig file>) tuples.
+ #
+ # <include path> is immutable and holds a *tuple* of
+ # (<filename>, <linenr>) tuples, giving the locations of the 'source'
+ # statements in the parent Kconfig files. The current include path is
+ # also available in Kconfig._include_path.
+ #
+ # The point of this redundant setup is to allow Kconfig._include_path
+ # to be assigned directly to MenuNode.include_path without having to
+ # copy it, sharing it wherever possible.
+
+ # Save include path and 'file' object (via its 'readline' function)
+ # before entering the file
+ self._filestack.append((self._include_path, self._readline))
+
+ # _include_path is a tuple, so this rebinds the variable instead of
+ # doing in-place modification
+ self._include_path += ((self.filename, self.linenr),)
+
+ # Check for recursive 'source'
+ for name, _ in self._include_path:
+ if name == rel_filename:
+ raise KconfigError(
+ "\n{}:{}: recursive 'source' of '{}' detected. Check that "
+ "environment variables are set correctly.\n"
+ "Include path:\n{}"
+ .format(self.filename, self.linenr, rel_filename,
+ "\n".join("{}:{}".format(name, linenr)
+ for name, linenr in self._include_path)))
+
+ try:
+ self._readline = self._open(filename, "r").readline
+ except EnvironmentError as e:
+ # We already know that the file exists
+ raise _KconfigIOError(
+ e, "{}:{}: Could not open '{}' (in '{}') ({}: {})"
+ .format(self.filename, self.linenr, filename,
+ self._line.strip(),
+ errno.errorcode[e.errno], e.strerror))
+
+ self.filename = rel_filename
+ self.linenr = 0
+
+ def _leave_file(self):
+ # Returns from a Kconfig file to the file that sourced it. See
+ # _enter_file().
+
+ # Restore location from parent Kconfig file
+ self.filename, self.linenr = self._include_path[-1]
+ # Restore include path and 'file' object
+ self._readline.__self__.close() # __self__ fetches the 'file' object
+ self._include_path, self._readline = self._filestack.pop()
+
+ def _next_line(self):
+ # Fetches and tokenizes the next line from the current Kconfig file.
+ # Returns False at EOF and True otherwise.
+
+ # We might already have tokens from parsing a line and discovering that
+ # it's part of a different construct
+ if self._reuse_tokens:
+ self._reuse_tokens = False
+ # self._tokens_i is known to be 1 here, because _parse_props()
+ # leaves it like that when it can't recognize a line (or parses a
+ # help text)
+ return True
+
+ # readline() returns '' over and over at EOF, which we rely on for help
+ # texts at the end of files (see _line_after_help())
+ line = self._readline()
+ if not line:
+ return False
+ self.linenr += 1
+
+ # Handle line joining
+ while line.endswith("\\\n"):
+ line = line[:-2] + self._readline()
+ self.linenr += 1
+
+ self._tokens = self._tokenize(line)
+ # Initialize to 1 instead of 0 to factor out code from _parse_block()
+ # and _parse_props(). They immediately fetch self._tokens[0].
+ self._tokens_i = 1
+
+ return True
+
+ def _line_after_help(self, line):
+ # Tokenizes a line after a help text. This case is special in that the
+ # line has already been fetched (to discover that it isn't part of the
+ # help text).
+ #
+ # An earlier version used a _saved_line variable instead that was
+ # checked in _next_line(). This special-casing gets rid of it and makes
+ # _reuse_tokens alone sufficient to handle unget.
+
+ # Handle line joining
+ while line.endswith("\\\n"):
+ line = line[:-2] + self._readline()
+ self.linenr += 1
+
+ self._tokens = self._tokenize(line)
+ self._reuse_tokens = True
+
+ def _write_if_changed(self, filename, contents):
+ # Writes 'contents' into 'filename', but only if it differs from the
+ # current contents of the file.
+ #
+ # Another variant would be write a temporary file on the same
+ # filesystem, compare the files, and rename() the temporary file if it
+ # differs, but it breaks stuff like write_config("/dev/null"), which is
+ # used out there to force evaluation-related warnings to be generated.
+ # This simple version is pretty failsafe and portable.
+ #
+ # Returns True if the file has changed and is updated, and False
+ # otherwise.
+
+ if self._contents_eq(filename, contents):
+ return False
+ with self._open(filename, "w") as f:
+ f.write(contents)
+ return True
+
+ def _contents_eq(self, filename, contents):
+ # Returns True if the contents of 'filename' is 'contents' (a string),
+ # and False otherwise (including if 'filename' can't be opened/read)
+
+ try:
+ with self._open(filename, "r") as f:
+ # Robust re. things like encoding and line endings (mmap()
+ # trickery isn't)
+ return f.read(len(contents) + 1) == contents
+ except EnvironmentError:
+ # If the error here would prevent writing the file as well, we'll
+ # notice it later
+ return False
+
+ #
+ # Tokenization
+ #
+
+ def _lookup_sym(self, name):
+ # Fetches the symbol 'name' from the symbol table, creating and
+ # registering it if it does not exist. If '_parsing_kconfigs' is False,
+ # it means we're in eval_string(), and new symbols won't be registered.
+
+ if name in self.syms:
+ return self.syms[name]
+
+ sym = Symbol()
+ sym.kconfig = self
+ sym.name = name
+ sym.is_constant = False
+ sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
+
+ if self._parsing_kconfigs:
+ self.syms[name] = sym
+ else:
+ self._warn("no symbol {} in configuration".format(name))
+
+ return sym
+
+ def _lookup_const_sym(self, name):
+ # Like _lookup_sym(), for constant (quoted) symbols
+
+ if name in self.const_syms:
+ return self.const_syms[name]
+
+ sym = Symbol()
+ sym.kconfig = self
+ sym.name = name
+ sym.is_constant = True
+ sym.rev_dep = sym.weak_rev_dep = sym.direct_dep = self.n
+
+ if self._parsing_kconfigs:
+ self.const_syms[name] = sym
+
+ return sym
+
+ def _tokenize(self, s):
+ # Parses 's', returning a None-terminated list of tokens. Registers any
+ # new symbols encountered with _lookup(_const)_sym().
+ #
+ # Tries to be reasonably speedy by processing chunks of text via
+ # regexes and string operations where possible. This is the biggest
+ # hotspot during parsing.
+ #
+ # It might be possible to rewrite this to 'yield' tokens instead,
+ # working across multiple lines. Lookback and compatibility with old
+ # janky versions of the C tools complicate things though.
+
+ self._line = s # Used for error reporting
+
+ # Initial token on the line
+ match = _command_match(s)
+ if not match:
+ if s.isspace() or s.lstrip().startswith("#"):
+ return (None,)
+ self._parse_error("unknown token at start of line")
+
+ # Tricky implementation detail: While parsing a token, 'token' refers
+ # to the previous token. See _STRING_LEX for why this is needed.
+ token = _get_keyword(match.group(1))
+ if not token:
+ # Backwards compatibility with old versions of the C tools, which
+ # (accidentally) accepted stuff like "--help--" and "-help---".
+ # This was fixed in the C tools by commit c2264564 ("kconfig: warn
+ # of unhandled characters in Kconfig commands"), committed in July
+ # 2015, but it seems people still run Kconfiglib on older kernels.
+ if s.strip(" \t\n-") == "help":
+ return (_T_HELP, None)
+
+ # If the first token is not a keyword (and not a weird help token),
+ # we have a preprocessor variable assignment (or a bare macro on a
+ # line)
+ self._parse_assignment(s)
+ return (None,)
+
+ tokens = [token]
+ # The current index in the string being tokenized
+ i = match.end()
+
+ # Main tokenization loop (for tokens past the first one)
+ while i < len(s):
+ # Test for an identifier/keyword first. This is the most common
+ # case.
+ match = _id_keyword_match(s, i)
+ if match:
+ # We have an identifier or keyword
+
+ # Check what it is. lookup_sym() will take care of allocating
+ # new symbols for us the first time we see them. Note that
+ # 'token' still refers to the previous token.
+
+ name = match.group(1)
+ keyword = _get_keyword(name)
+ if keyword:
+ # It's a keyword
+ token = keyword
+ # Jump past it
+ i = match.end()
+
+ elif token not in _STRING_LEX:
+ # It's a non-const symbol, except we translate n, m, and y
+ # into the corresponding constant symbols, like the C
+ # implementation
+
+ if "$" in name:
+ # Macro expansion within symbol name
+ name, s, i = self._expand_name(s, i)
+ else:
+ i = match.end()
+
+ token = self.const_syms[name] if name in STR_TO_TRI else \
+ self._lookup_sym(name)
+
+ else:
+ # It's a case of missing quotes. For example, the
+ # following is accepted:
+ #
+ # menu unquoted_title
+ #
+ # config A
+ # tristate unquoted_prompt
+ #
+ # endmenu
+ #
+ # Named choices ('choice FOO') also end up here.
+
+ if token not in (_T_CHOICE, _T_CONTCHOICE):
+ self._warn("style: quotes recommended around '{}' in '{}'"
+ .format(name, self._line.strip()),
+ self.filename, self.linenr)
+
+ token = name
+ i = match.end()
+
+ else:
+ # Neither a keyword nor a non-const symbol
+
+ # We always strip whitespace after tokens, so it is safe to
+ # assume that s[i] is the start of a token here.
+ c = s[i]
+
+ if c in "\"'":
+ if "$" not in s and "\\" not in s:
+ # Fast path for lines without $ and \. Find the
+ # matching quote.
+ end_i = s.find(c, i + 1) + 1
+ if not end_i:
+ self._parse_error("unterminated string")
+ val = s[i + 1:end_i - 1]
+ i = end_i
+ else:
+ # Slow path
+ s, end_i = self._expand_str(s, i)
+
+ # os.path.expandvars() and the $UNAME_RELEASE replace()
+ # is a backwards compatibility hack, which should be
+ # reasonably safe as expandvars() leaves references to
+ # undefined env. vars. as is.
+ #
+ # The preprocessor functionality changed how
+ # environment variables are referenced, to $(FOO).
+ val = expandvars(s[i + 1:end_i - 1]
+ .replace("$UNAME_RELEASE",
+ _UNAME_RELEASE))
+
+ i = end_i
+
+ # This is the only place where we don't survive with a
+ # single token of lookback: 'option env="FOO"' does not
+ # refer to a constant symbol named "FOO".
+ token = \
+ val if token in _STRING_LEX or tokens[0] is _T_OPTION \
+ else self._lookup_const_sym(val)
+
+ elif s.startswith("&&", i):
+ token = _T_AND
+ i += 2
+
+ elif s.startswith("||", i):
+ token = _T_OR
+ i += 2
+
+ elif c == "=":
+ token = _T_EQUAL
+ i += 1
+
+ elif s.startswith("!=", i):
+ token = _T_UNEQUAL
+ i += 2
+
+ elif c == "!":
+ token = _T_NOT
+ i += 1
+
+ elif c == "(":
+ token = _T_OPEN_PAREN
+ i += 1
+
+ elif c == ")":
+ token = _T_CLOSE_PAREN
+ i += 1
+
+ elif c == "#":
+ break
+
+
+ # Very rare
+
+ elif s.startswith("<=", i):
+ token = _T_LESS_EQUAL
+ i += 2
+
+ elif c == "<":
+ token = _T_LESS
+ i += 1
+
+ elif s.startswith(">=", i):
+ token = _T_GREATER_EQUAL
+ i += 2
+
+ elif c == ">":
+ token = _T_GREATER
+ i += 1
+
+
+ else:
+ self._parse_error("unknown tokens in line")
+
+
+ # Skip trailing whitespace
+ while i < len(s) and s[i].isspace():
+ i += 1
+
+
+ # Add the token
+ tokens.append(token)
+
+ # None-terminating the token list makes token fetching simpler/faster
+ tokens.append(None)
+
+ return tokens
+
+ # Helpers for syntax checking and token fetching. See the
+ # 'Intro to expressions' section for what a constant symbol is.
+ #
+ # More of these could be added, but the single-use cases are inlined as an
+ # optimization.
+
+ def _expect_sym(self):
+ token = self._tokens[self._tokens_i]
+ self._tokens_i += 1
+
+ if token.__class__ is not Symbol:
+ self._parse_error("expected symbol")
+
+ return token
+
+ def _expect_nonconst_sym(self):
+ # Used for 'select' and 'imply' only. We know the token indices.
+
+ token = self._tokens[1]
+ self._tokens_i = 2
+
+ if token.__class__ is not Symbol or token.is_constant:
+ self._parse_error("expected nonconstant symbol")
+
+ return token
+
+ def _expect_str_and_eol(self):
+ token = self._tokens[self._tokens_i]
+ self._tokens_i += 1
+
+ if token.__class__ is not str:
+ self._parse_error("expected string")
+
+ if self._tokens[self._tokens_i] is not None:
+ self._trailing_tokens_error()
+
+ return token
+
+ def _expect_expr_and_eol(self):
+ expr = self._parse_expr(True)
+
+ if self._tokens[self._tokens_i] is not None:
+ self._trailing_tokens_error()
+
+ return expr
+
+ def _check_token(self, token):
+ # If the next token is 'token', removes it and returns True
+
+ if self._tokens[self._tokens_i] is token:
+ self._tokens_i += 1
+ return True
+ return False
+
+ #
+ # Preprocessor logic
+ #
+
+ def _parse_assignment(self, s):
+ # Parses a preprocessor variable assignment, registering the variable
+ # if it doesn't already exist. Also takes care of bare macros on lines
+ # (which are allowed, and can be useful for their side effects).
+
+ # Expand any macros in the left-hand side of the assignment (the
+ # variable name)
+ s = s.lstrip()
+ i = 0
+ while 1:
+ i = _assignment_lhs_fragment_match(s, i).end()
+ if s.startswith("$(", i):
+ s, i = self._expand_macro(s, i, ())
+ else:
+ break
+
+ if s.isspace():
+ # We also accept a bare macro on a line (e.g.
+ # $(warning-if,$(foo),ops)), provided it expands to a blank string
+ return
+
+ # Assigned variable
+ name = s[:i]
+
+
+ # Extract assignment operator (=, :=, or +=) and value
+ rhs_match = _assignment_rhs_match(s, i)
+ if not rhs_match:
+ self._parse_error("syntax error")
+
+ op, val = rhs_match.groups()
+
+
+ if name in self.variables:
+ # Already seen variable
+ var = self.variables[name]
+ else:
+ # New variable
+ var = Variable()
+ var.kconfig = self
+ var.name = name
+ var._n_expansions = 0
+ self.variables[name] = var
+
+ # += acts like = on undefined variables (defines a recursive
+ # variable)
+ if op == "+=":
+ op = "="
+
+ if op == "=":
+ var.is_recursive = True
+ var.value = val
+ elif op == ":=":
+ var.is_recursive = False
+ var.value = self._expand_whole(val, ())
+ else: # op == "+="
+ # += does immediate expansion if the variable was last set
+ # with :=
+ var.value += " " + (val if var.is_recursive else
+ self._expand_whole(val, ()))
+
+ def _expand_whole(self, s, args):
+ # Expands preprocessor macros in all of 's'. Used whenever we don't
+ # have to worry about delimiters. See _expand_macro() re. the 'args'
+ # parameter.
+ #
+ # Returns the expanded string.
+
+ i = 0
+ while 1:
+ i = s.find("$(", i)
+ if i == -1:
+ break
+ s, i = self._expand_macro(s, i, args)
+ return s
+
+ def _expand_name(self, s, i):
+ # Expands a symbol name starting at index 'i' in 's'.
+ #
+ # Returns the expanded name, the expanded 's' (including the part
+ # before the name), and the index of the first character in the next
+ # token after the name.
+
+ s, end_i = self._expand_name_iter(s, i)
+ name = s[i:end_i]
+ # isspace() is False for empty strings
+ if not name.strip():
+ # Avoid creating a Kconfig symbol with a blank name. It's almost
+ # guaranteed to be an error.
+ self._parse_error("macro expanded to blank string")
+
+ # Skip trailing whitespace
+ while end_i < len(s) and s[end_i].isspace():
+ end_i += 1
+
+ return name, s, end_i
+
+ def _expand_name_iter(self, s, i):
+ # Expands a symbol name starting at index 'i' in 's'.
+ #
+ # Returns the expanded 's' (including the part before the name) and the
+ # index of the first character after the expanded name in 's'.
+
+ while 1:
+ match = _name_special_search(s, i)
+
+ if match.group() != "$(":
+ return (s, match.start())
+ s, i = self._expand_macro(s, match.start(), ())
+
+ def _expand_str(self, s, i):
+ # Expands a quoted string starting at index 'i' in 's'. Handles both
+ # backslash escapes and macro expansion.
+ #
+ # Returns the expanded 's' (including the part before the string) and
+ # the index of the first character after the expanded string in 's'.
+
+ quote = s[i]
+ i += 1 # Skip over initial "/'
+ while 1:
+ match = _string_special_search(s, i)
+ if not match:
+ self._parse_error("unterminated string")
+
+
+ if match.group() == quote:
+ # Found the end of the string
+ return (s, match.end())
+
+ elif match.group() == "\\":
+ # Replace '\x' with 'x'. 'i' ends up pointing to the character
+ # after 'x', which allows macros to be canceled with '\$(foo)'.
+ i = match.end()
+ s = s[:match.start()] + s[i:]
+
+ elif match.group() == "$(":
+ # A macro call within the string
+ s, i = self._expand_macro(s, match.start(), ())
+
+ else:
+ # A ' quote within " quotes or vice versa
+ i += 1
+
+ def _expand_macro(self, s, i, args):
+ # Expands a macro starting at index 'i' in 's'. If this macro resulted
+ # from the expansion of another macro, 'args' holds the arguments
+ # passed to that macro.
+ #
+ # Returns the expanded 's' (including the part before the macro) and
+ # the index of the first character after the expanded macro in 's'.
+
+ res = s[:i]
+ i += 2 # Skip over "$("
+
+ arg_start = i # Start of current macro argument
+ new_args = [] # Arguments of this macro call
+ nesting = 0 # Current parentheses nesting level
+
+ while 1:
+ match = _macro_special_search(s, i)
+ if not match:
+ self._parse_error("missing end parenthesis in macro expansion")
+
+
+ if match.group() == "(":
+ nesting += 1
+ i = match.end()
+
+ elif match.group() == ")":
+ if nesting:
+ nesting -= 1
+ i = match.end()
+ continue
+
+ # Found the end of the macro
+
+ new_args.append(s[arg_start:match.start()])
+
+ # $(1) is replaced by the first argument to the function, etc.,
+ # provided at least that many arguments were passed
+
+ try:
+ # Does the macro look like an integer, with a corresponding
+ # argument? If so, expand it to the value of the argument.
+ res += args[int(new_args[0])]
+ except (ValueError, IndexError):
+ # Regular variables are just functions without arguments,
+ # and also go through the function value path
+ res += self._fn_val(new_args)
+
+ return (res + s[match.end():], len(res))
+
+ elif match.group() == ",":
+ i = match.end()
+ if nesting:
+ continue
+
+ # Found the end of a macro argument
+ new_args.append(s[arg_start:match.start()])
+ arg_start = i
+
+ else: # match.group() == "$("
+ # A nested macro call within the macro
+ s, i = self._expand_macro(s, match.start(), args)
+
+ def _fn_val(self, args):
+ # Returns the result of calling the function args[0] with the arguments
+ # args[1..len(args)-1]. Plain variables are treated as functions
+ # without arguments.
+
+ fn = args[0]
+
+ if fn in self.variables:
+ var = self.variables[fn]
+
+ if len(args) == 1:
+ # Plain variable
+ if var._n_expansions:
+ self._parse_error("Preprocessor variable {} recursively "
+ "references itself".format(var.name))
+ elif var._n_expansions > 100:
+ # Allow functions to call themselves, but guess that functions
+ # that are overly recursive are stuck
+ self._parse_error("Preprocessor function {} seems stuck "
+ "in infinite recursion".format(var.name))
+
+ var._n_expansions += 1
+ res = self._expand_whole(self.variables[fn].value, args)
+ var._n_expansions -= 1
+ return res
+
+ if fn in self._functions:
+ # Built-in or user-defined function
+
+ py_fn, min_arg, max_arg = self._functions[fn]
+
+ if len(args) - 1 < min_arg or \
+ (max_arg is not None and len(args) - 1 > max_arg):
+
+ if min_arg == max_arg:
+ expected_args = min_arg
+ elif max_arg is None:
+ expected_args = "{} or more".format(min_arg)
+ else:
+ expected_args = "{}-{}".format(min_arg, max_arg)
+
+ raise KconfigError("{}:{}: bad number of arguments in call "
+ "to {}, expected {}, got {}"
+ .format(self.filename, self.linenr, fn,
+ expected_args, len(args) - 1))
+
+ return py_fn(self, *args)
+
+ # Environment variables are tried last
+ if fn in os.environ:
+ self.env_vars.add(fn)
+ return os.environ[fn]
+
+ return ""
+
+ #
+ # Parsing
+ #
+
+ def _make_and(self, e1, e2):
+ # Constructs an AND (&&) expression. Performs trivial simplification.
+
+ if e1 is self.y:
+ return e2
+
+ if e2 is self.y:
+ return e1
+
+ if e1 is self.n or e2 is self.n:
+ return self.n
+
+ return (AND, e1, e2)
+
+ def _make_or(self, e1, e2):
+ # Constructs an OR (||) expression. Performs trivial simplification.
+
+ if e1 is self.n:
+ return e2
+
+ if e2 is self.n:
+ return e1
+
+ if e1 is self.y or e2 is self.y:
+ return self.y
+
+ return (OR, e1, e2)
+
+ def _parse_block(self, end_token, parent, prev):
+ # Parses a block, which is the contents of either a file or an if,
+ # menu, or choice statement.
+ #
+ # end_token:
+ # The token that ends the block, e.g. _T_ENDIF ("endif") for ifs.
+ # None for files.
+ #
+ # parent:
+ # The parent menu node, corresponding to a menu, Choice, or 'if'.
+ # 'if's are flattened after parsing.
+ #
+ # prev:
+ # The previous menu node. New nodes will be added after this one (by
+ # modifying 'next' pointers).
+ #
+ # 'prev' is reused to parse a list of child menu nodes (for a menu or
+ # Choice): After parsing the children, the 'next' pointer is assigned
+ # to the 'list' pointer to "tilt up" the children above the node.
+ #
+ # Returns the final menu node in the block (or 'prev' if the block is
+ # empty). This allows chaining.
+
+ while self._next_line():
+ t0 = self._tokens[0]
+
+ if t0 is _T_CONFIG or t0 is _T_MENUCONFIG:
+ # The tokenizer allocates Symbol objects for us
+ sym = self._tokens[1]
+
+ if sym.__class__ is not Symbol or sym.is_constant:
+ self._parse_error("missing or bad symbol name")
+
+ if self._tokens[2] is not None:
+ self._trailing_tokens_error()
+
+ self.defined_syms.append(sym)
+
+ node = MenuNode()
+ node.kconfig = self
+ node.item = sym
+ node.is_menuconfig = (t0 is _T_MENUCONFIG)
+ node.prompt = node.help = node.list = None
+ node.parent = parent
+ node.filename = self.filename
+ node.linenr = self.linenr
+ node.include_path = self._include_path
+
+ sym.nodes.append(node)
+
+ self._parse_props(node)
+
+ if node.is_menuconfig and not node.prompt:
+ self._warn("the menuconfig symbol {} has no prompt"
+ .format(sym.name_and_loc))
+
+ # Equivalent to
+ #
+ # prev.next = node
+ # prev = node
+ #
+ # due to tricky Python semantics. The order matters.
+ prev.next = prev = node
+
+ elif t0 is None:
+ # Blank line
+ continue
+
+ elif t0 in _SOURCE_TOKENS:
+ pattern = self._expect_str_and_eol()
+
+ if t0 in _REL_SOURCE_TOKENS:
+ # Relative source
+ pattern = join(dirname(self.filename), pattern)
+
+ # - glob() doesn't support globbing relative to a directory, so
+ # we need to prepend $srctree to 'pattern'. Use join()
+ # instead of '+' so that an absolute path in 'pattern' is
+ # preserved.
+ #
+ # - Sort the glob results to ensure a consistent ordering of
+ # Kconfig symbols, which indirectly ensures a consistent
+ # ordering in e.g. .config files
+ filenames = sorted(iglob(join(self._srctree_prefix, pattern)))
+
+ if not filenames and t0 in _OBL_SOURCE_TOKENS:
+ raise KconfigError(
+ "{}:{}: '{}' not found (in '{}'). Check that "
+ "environment variables are set correctly (e.g. "
+ "$srctree, which is {}). Also note that unset "
+ "environment variables expand to the empty string."
+ .format(self.filename, self.linenr, pattern,
+ self._line.strip(),
+ "set to '{}'".format(self.srctree)
+ if self.srctree else "unset or blank"))
+
+ for filename in filenames:
+ self._enter_file(filename)
+ prev = self._parse_block(None, parent, prev)
+ self._leave_file()
+
+ elif t0 is end_token:
+ # Reached the end of the block. Terminate the final node and
+ # return it.
+
+ if self._tokens[1] is not None:
+ self._trailing_tokens_error()
+
+ prev.next = None
+ return prev
+
+ elif t0 is _T_IF:
+ node = MenuNode()
+ node.item = node.prompt = None
+ node.parent = parent
+ node.dep = self._expect_expr_and_eol()
+
+ self._parse_block(_T_ENDIF, node, node)
+ node.list = node.next
+
+ prev.next = prev = node
+
+ elif t0 is _T_MENU:
+ node = MenuNode()
+ node.kconfig = self
+ node.item = t0 # _T_MENU == MENU
+ node.is_menuconfig = True
+ node.prompt = (self._expect_str_and_eol(), self.y)
+ node.visibility = self.y
+ node.parent = parent
+ node.filename = self.filename
+ node.linenr = self.linenr
+ node.include_path = self._include_path
+
+ self.menus.append(node)
+
+ self._parse_props(node)
+ self._parse_block(_T_ENDMENU, node, node)
+ node.list = node.next
+
+ prev.next = prev = node
+
+ elif t0 is _T_COMMENT:
+ node = MenuNode()
+ node.kconfig = self
+ node.item = t0 # _T_COMMENT == COMMENT
+ node.is_menuconfig = False
+ node.prompt = (self._expect_str_and_eol(), self.y)
+ node.list = None
+ node.parent = parent
+ node.filename = self.filename
+ node.linenr = self.linenr
+ node.include_path = self._include_path
+
+ self.comments.append(node)
+
+ self._parse_props(node)
+
+ prev.next = prev = node
+
+ elif t0 is _T_CHOICE:
+ if self._tokens[1] is None:
+ choice = Choice()
+ choice.direct_dep = self.n
+ else:
+ # Named choice
+ name = self._expect_str_and_eol()
+ choice = self.named_choices.get(name)
+ if not choice:
+ choice = Choice()
+ choice.name = name
+ choice.direct_dep = self.n
+ self.named_choices[name] = choice
+
+ self.choices.append(choice)
+
+ node = MenuNode()
+ node.kconfig = choice.kconfig = self
+ node.item = choice
+ node.is_menuconfig = True
+ node.prompt = node.help = None
+ node.parent = parent
+ node.filename = self.filename
+ node.linenr = self.linenr
+ node.include_path = self._include_path
+
+ choice.nodes.append(node)
+
+ self._parse_props(node)
+ self._parse_block(_T_ENDCHOICE, node, node)
+
+ node.list = node.next
+ prev.next = prev = node
+
+ elif t0 is _T_CONTCHOICE:
+ # Named choice
+ name = self._expect_str_and_eol()
+ choice = self.named_choices.get(name)
+ if not choice:
+ self._parse_error(f"can't continue choice '{name}'")
+
+ assert(len(choice.nodes))
+ # Add more to the earlier node.
+ node = choice.nodes[-1]
+
+ # Find the end of its list so we can add to it.
+ if node.list:
+ sub_prev = node.list
+ while sub_prev.next:
+ sub_prev = sub_prev.next
+ else:
+ # If we don't have a list at all, temporarily make one up.
+ sub_prev = MenuNode()
+
+ # Parse any new properties.
+ self._parse_props(node)
+ # Read in new subnodes.
+ self._parse_block(_T_ENDCHOICE, node, sub_prev)
+
+ # If we made up a lead node, move the list to where it belongs.
+ if not node.list:
+ node.list = sub_prev.next
+
+ elif t0 is _T_MAINMENU:
+ self.top_node.prompt = (self._expect_str_and_eol(), self.y)
+
+ else:
+ # A valid endchoice/endif/endmenu is caught by the 'end_token'
+ # check above
+ self._parse_error(
+ "no corresponding 'choice'" if t0 is _T_ENDCHOICE else
+ "no corresponding 'if'" if t0 is _T_ENDIF else
+ "no corresponding 'menu'" if t0 is _T_ENDMENU else
+ "unrecognized construct")
+
+ # End of file reached. Return the last node.
+
+ if end_token:
+ raise KconfigError(
+ "error: expected '{}' at end of '{}'"
+ .format("endchoice" if end_token is _T_ENDCHOICE else
+ "endif" if end_token is _T_ENDIF else
+ "endmenu",
+ self.filename))
+
+ return prev
+
+ def _parse_cond(self):
+ # Parses an optional 'if <expr>' construct and returns the parsed
+ # <expr>, or self.y if the next token is not _T_IF
+
+ expr = self._parse_expr(True) if self._check_token(_T_IF) else self.y
+
+ if self._tokens[self._tokens_i] is not None:
+ self._trailing_tokens_error()
+
+ return expr
+
+ def _parse_props(self, node):
+ # Parses and adds properties to the MenuNode 'node' (type, 'prompt',
+ # 'default's, etc.) Properties are later copied up to symbols and
+ # choices in a separate pass after parsing, in e.g.
+ # _add_props_to_sym().
+ #
+ # An older version of this code added properties directly to symbols
+ # and choices instead of to their menu nodes (and handled dependency
+ # propagation simultaneously), but that loses information on where a
+ # property is added when a symbol or choice is defined in multiple
+ # locations. Some Kconfig configuration systems rely heavily on such
+ # symbols, and better docs can be generated by keeping track of where
+ # properties are added.
+ #
+ # node:
+ # The menu node we're parsing properties on
+
+ # Dependencies from 'depends on'. Will get propagated to the properties
+ # below.
+ node.dep = self.y
+
+ while self._next_line():
+ t0 = self._tokens[0]
+
+ if t0 in _TYPE_TOKENS:
+ # Relies on '_T_BOOL is BOOL', etc., to save a conversion
+ self._set_type(node.item, t0)
+ if self._tokens[1] is not None:
+ self._parse_prompt(node)
+
+ elif t0 is _T_DEPENDS:
+ if not self._check_token(_T_ON):
+ self._parse_error("expected 'on' after 'depends'")
+
+ node.dep = self._make_and(node.dep,
+ self._expect_expr_and_eol())
+
+ elif t0 is _T_HELP:
+ self._parse_help(node)
+
+ elif t0 is _T_SELECT:
+ if node.item.__class__ is not Symbol:
+ self._parse_error("only symbols can select")
+
+ node.selects.append((self._expect_nonconst_sym(),
+ self._parse_cond()))
+
+ elif t0 is None:
+ # Blank line
+ continue
+
+ elif t0 is _T_DEFAULT:
+ node.defaults.append((self._parse_expr(False),
+ self._parse_cond()))
+
+ elif t0 in _DEF_TOKEN_TO_TYPE:
+ self._set_type(node.item, _DEF_TOKEN_TO_TYPE[t0])
+ node.defaults.append((self._parse_expr(False),
+ self._parse_cond()))
+
+ elif t0 is _T_PROMPT:
+ self._parse_prompt(node)
+
+ elif t0 is _T_RANGE:
+ node.ranges.append((self._expect_sym(), self._expect_sym(),
+ self._parse_cond()))
+
+ elif t0 is _T_IMPLY:
+ if node.item.__class__ is not Symbol:
+ self._parse_error("only symbols can imply")
+
+ node.implies.append((self._expect_nonconst_sym(),
+ self._parse_cond()))
+
+ elif t0 is _T_VISIBLE:
+ if not self._check_token(_T_IF):
+ self._parse_error("expected 'if' after 'visible'")
+
+ node.visibility = self._make_and(node.visibility,
+ self._expect_expr_and_eol())
+
+ elif t0 is _T_OPTION:
+ if self._check_token(_T_ENV):
+ if not self._check_token(_T_EQUAL):
+ self._parse_error("expected '=' after 'env'")
+
+ env_var = self._expect_str_and_eol()
+ node.item.env_var = env_var
+
+ if env_var in os.environ:
+ node.defaults.append(
+ (self._lookup_const_sym(os.environ[env_var]),
+ self.y))
+ else:
+ self._warn("{1} has 'option env=\"{0}\"', "
+ "but the environment variable {0} is not "
+ "set".format(node.item.name, env_var),
+ self.filename, self.linenr)
+
+ if env_var != node.item.name:
+ self._warn("Kconfiglib expands environment variables "
+ "in strings directly, meaning you do not "
+ "need 'option env=...' \"bounce\" symbols. "
+ "For compatibility with the C tools, "
+ "rename {} to {} (so that the symbol name "
+ "matches the environment variable name)."
+ .format(node.item.name, env_var),
+ self.filename, self.linenr)
+
+ elif self._check_token(_T_DEFCONFIG_LIST):
+ if not self.defconfig_list:
+ self.defconfig_list = node.item
+ else:
+ self._warn("'option defconfig_list' set on multiple "
+ "symbols ({0} and {1}). Only {0} will be "
+ "used.".format(self.defconfig_list.name,
+ node.item.name),
+ self.filename, self.linenr)
+
+ elif self._check_token(_T_MODULES):
+ # To reduce warning spam, only warn if 'option modules' is
+ # set on some symbol that isn't MODULES, which should be
+ # safe. I haven't run into any projects that make use
+ # modules besides the kernel yet, and there it's likely to
+ # keep being called "MODULES".
+ if node.item is not self.modules:
+ self._warn("the 'modules' option is not supported. "
+ "Let me know if this is a problem for you, "
+ "as it wouldn't be that hard to implement. "
+ "Note that modules are supported -- "
+ "Kconfiglib just assumes the symbol name "
+ "MODULES, like older versions of the C "
+ "implementation did when 'option modules' "
+ "wasn't used.",
+ self.filename, self.linenr)
+
+ elif self._check_token(_T_ALLNOCONFIG_Y):
+ if node.item.__class__ is not Symbol:
+ self._parse_error("the 'allnoconfig_y' option is only "
+ "valid for symbols")
+
+ node.item.is_allnoconfig_y = True
+
+ else:
+ self._parse_error("unrecognized option")
+
+ elif t0 is _T_OPTIONAL:
+ if node.item.__class__ is not Choice:
+ self._parse_error('"optional" is only valid for choices')
+
+ node.item.is_optional = True
+
+ else:
+ # Reuse the tokens for the non-property line later
+ self._reuse_tokens = True
+ return
+
+ def _set_type(self, sc, new_type):
+ # Sets the type of 'sc' (symbol or choice) to 'new_type'
+
+ # UNKNOWN is falsy
+ if sc.orig_type and sc.orig_type is not new_type:
+ self._warn("{} defined with multiple types, {} will be used"
+ .format(sc.name_and_loc, TYPE_TO_STR[new_type]))
+
+ sc.orig_type = new_type
+
+ def _parse_prompt(self, node):
+ # 'prompt' properties override each other within a single definition of
+ # a symbol, but additional prompts can be added by defining the symbol
+ # multiple times
+
+ if node.prompt:
+ self._warn(node.item.name_and_loc +
+ " defined with multiple prompts in single location")
+
+ prompt = self._tokens[1]
+ self._tokens_i = 2
+
+ if prompt.__class__ is not str:
+ self._parse_error("expected prompt string")
+
+ if prompt != prompt.strip():
+ self._warn(node.item.name_and_loc +
+ " has leading or trailing whitespace in its prompt")
+
+ # This avoid issues for e.g. reStructuredText documentation, where
+ # '*prompt *' is invalid
+ prompt = prompt.strip()
+
+ node.prompt = (prompt, self._parse_cond())
+
+ def _parse_help(self, node):
+ if node.help is not None:
+ self._warn(node.item.name_and_loc + " defined with more than "
+ "one help text -- only the last one will be used")
+
+ # Micro-optimization. This code is pretty hot.
+ readline = self._readline
+
+ # Find first non-blank (not all-space) line and get its
+ # indentation
+
+ while 1:
+ line = readline()
+ self.linenr += 1
+ if not line:
+ self._empty_help(node, line)
+ return
+ if not line.isspace():
+ break
+
+ len_ = len # Micro-optimization
+
+ # Use a separate 'expline' variable here and below to avoid stomping on
+ # any tabs people might've put deliberately into the first line after
+ # the help text
+ expline = line.expandtabs()
+ indent = len_(expline) - len_(expline.lstrip())
+ if not indent:
+ self._empty_help(node, line)
+ return
+
+ # The help text goes on till the first non-blank line with less indent
+ # than the first line
+
+ # Add the first line
+ lines = [expline[indent:]]
+ add_line = lines.append # Micro-optimization
+
+ while 1:
+ line = readline()
+ if line.isspace():
+ # No need to preserve the exact whitespace in these
+ add_line("\n")
+ elif not line:
+ # End of file
+ break
+ else:
+ expline = line.expandtabs()
+ if len_(expline) - len_(expline.lstrip()) < indent:
+ break
+ add_line(expline[indent:])
+
+ self.linenr += len_(lines)
+ node.help = "".join(lines).rstrip()
+ if line:
+ self._line_after_help(line)
+
+ def _empty_help(self, node, line):
+ self._warn(node.item.name_and_loc +
+ " has 'help' but empty help text")
+ node.help = ""
+ if line:
+ self._line_after_help(line)
+
+ def _parse_expr(self, transform_m):
+ # Parses an expression from the tokens in Kconfig._tokens using a
+ # simple top-down approach. See the module docstring for the expression
+ # format.
+ #
+ # transform_m:
+ # True if m should be rewritten to m && MODULES. See the
+ # Kconfig.eval_string() documentation.
+
+ # Grammar:
+ #
+ # expr: and_expr ['||' expr]
+ # and_expr: factor ['&&' and_expr]
+ # factor: <symbol> ['='/'!='/'<'/... <symbol>]
+ # '!' factor
+ # '(' expr ')'
+ #
+ # It helps to think of the 'expr: and_expr' case as a single-operand OR
+ # (no ||), and of the 'and_expr: factor' case as a single-operand AND
+ # (no &&). Parsing code is always a bit tricky.
+
+ # Mind dump: parse_factor() and two nested loops for OR and AND would
+ # work as well. The straightforward implementation there gives a
+ # (op, (op, (op, A, B), C), D) parse for A op B op C op D. Representing
+ # expressions as (op, [list of operands]) instead goes nicely with that
+ # version, but is wasteful for short expressions and complicates
+ # expression evaluation and other code that works on expressions (more
+ # complicated code likely offsets any performance gain from less
+ # recursion too). If we also try to optimize the list representation by
+ # merging lists when possible (e.g. when ANDing two AND expressions),
+ # we end up allocating a ton of lists instead of reusing expressions,
+ # which is bad.
+
+ and_expr = self._parse_and_expr(transform_m)
+
+ # Return 'and_expr' directly if we have a "single-operand" OR.
+ # Otherwise, parse the expression on the right and make an OR node.
+ # This turns A || B || C || D into (OR, A, (OR, B, (OR, C, D))).
+ return and_expr if not self._check_token(_T_OR) else \
+ (OR, and_expr, self._parse_expr(transform_m))
+
+ def _parse_and_expr(self, transform_m):
+ factor = self._parse_factor(transform_m)
+
+ # Return 'factor' directly if we have a "single-operand" AND.
+ # Otherwise, parse the right operand and make an AND node. This turns
+ # A && B && C && D into (AND, A, (AND, B, (AND, C, D))).
+ return factor if not self._check_token(_T_AND) else \
+ (AND, factor, self._parse_and_expr(transform_m))
+
+ def _parse_factor(self, transform_m):
+ token = self._tokens[self._tokens_i]
+ self._tokens_i += 1
+
+ if token.__class__ is Symbol:
+ # Plain symbol or relation
+
+ if self._tokens[self._tokens_i] not in _RELATIONS:
+ # Plain symbol
+
+ # For conditional expressions ('depends on <expr>',
+ # '... if <expr>', etc.), m is rewritten to m && MODULES.
+ if transform_m and token is self.m:
+ return (AND, self.m, self.modules)
+
+ return token
+
+ # Relation
+ #
+ # _T_EQUAL, _T_UNEQUAL, etc., deliberately have the same values as
+ # EQUAL, UNEQUAL, etc., so we can just use the token directly
+ self._tokens_i += 1
+ return (self._tokens[self._tokens_i - 1], token,
+ self._expect_sym())
+
+ if token is _T_NOT:
+ # token == _T_NOT == NOT
+ return (token, self._parse_factor(transform_m))
+
+ if token is _T_OPEN_PAREN:
+ expr_parse = self._parse_expr(transform_m)
+ if self._check_token(_T_CLOSE_PAREN):
+ return expr_parse
+
+ self._parse_error("malformed expression")
+
+ #
+ # Caching and invalidation
+ #
+
+ def _build_dep(self):
+ # Populates the Symbol/Choice._dependents sets, which contain all other
+ # items (symbols and choices) that immediately depend on the item in
+ # the sense that changing the value of the item might affect the value
+ # of the dependent items. This is used for caching/invalidation.
+ #
+ # The calculated sets might be larger than necessary as we don't do any
+ # complex analysis of the expressions.
+
+ depend_on = _depend_on # Micro-optimization
+
+ # Only calculate _dependents for defined symbols. Constant and
+ # undefined symbols could theoretically be selected/implied, but it
+ # wouldn't change their value, so it's not a true dependency.
+ for sym in self.unique_defined_syms:
+ # Symbols depend on the following:
+
+ # The prompt conditions
+ for node in sym.nodes:
+ if node.prompt:
+ depend_on(sym, node.prompt[1])
+
+ # The default values and their conditions
+ for value, cond in sym.defaults:
+ depend_on(sym, value)
+ depend_on(sym, cond)
+
+ # The reverse and weak reverse dependencies
+ depend_on(sym, sym.rev_dep)
+ depend_on(sym, sym.weak_rev_dep)
+
+ # The ranges along with their conditions
+ for low, high, cond in sym.ranges:
+ depend_on(sym, low)
+ depend_on(sym, high)
+ depend_on(sym, cond)
+
+ # The direct dependencies. This is usually redundant, as the direct
+ # dependencies get propagated to properties, but it's needed to get
+ # invalidation solid for 'imply', which only checks the direct
+ # dependencies (even if there are no properties to propagate it
+ # to).
+ depend_on(sym, sym.direct_dep)
+
+ # In addition to the above, choice symbols depend on the choice
+ # they're in, but that's handled automatically since the Choice is
+ # propagated to the conditions of the properties before
+ # _build_dep() runs.
+
+ for choice in self.unique_choices:
+ # Choices depend on the following:
+
+ # The prompt conditions
+ for node in choice.nodes:
+ if node.prompt:
+ depend_on(choice, node.prompt[1])
+
+ # The default symbol conditions
+ for _, cond in choice.defaults:
+ depend_on(choice, cond)
+
+ def _add_choice_deps(self):
+ # Choices also depend on the choice symbols themselves, because the
+ # y-mode selection of the choice might change if a choice symbol's
+ # visibility changes.
+ #
+ # We add these dependencies separately after dependency loop detection.
+ # The invalidation algorithm can handle the resulting
+ # <choice symbol> <-> <choice> dependency loops, but they make loop
+ # detection awkward.
+
+ for choice in self.unique_choices:
+ for sym in choice.syms:
+ sym._dependents.add(choice)
+
+ def _invalidate_all(self):
+ # Undefined symbols never change value and don't need to be
+ # invalidated, so we can just iterate over defined symbols.
+ # Invalidating constant symbols would break things horribly.
+ for sym in self.unique_defined_syms:
+ sym._invalidate()
+
+ for choice in self.unique_choices:
+ choice._invalidate()
+
+ #
+ # Post-parsing menu tree processing, including dependency propagation and
+ # implicit submenu creation
+ #
+
+ def _finalize_node(self, node, visible_if):
+ # Finalizes a menu node and its children:
+ #
+ # - Copies properties from menu nodes up to their contained
+ # symbols/choices
+ #
+ # - Propagates dependencies from parent to child nodes
+ #
+ # - Creates implicit menus (see kconfig-language.txt)
+ #
+ # - Removes 'if' nodes
+ #
+ # - Sets 'choice' types and registers choice symbols
+ #
+ # menu_finalize() in the C implementation is similar.
+ #
+ # node:
+ # The menu node to finalize. This node and its children will have
+ # been finalized when the function returns, and any implicit menus
+ # will have been created.
+ #
+ # visible_if:
+ # Dependencies from 'visible if' on parent menus. These are added to
+ # the prompts of symbols and choices.
+
+ if node.item.__class__ is Symbol:
+ # Copy defaults, ranges, selects, and implies to the Symbol
+ self._add_props_to_sym(node)
+
+ # Find any items that should go in an implicit menu rooted at the
+ # symbol
+ cur = node
+ while cur.next and _auto_menu_dep(node, cur.next):
+ # This makes implicit submenu creation work recursively, with
+ # implicit menus inside implicit menus
+ self._finalize_node(cur.next, visible_if)
+ cur = cur.next
+ cur.parent = node
+
+ if cur is not node:
+ # Found symbols that should go in an implicit submenu. Tilt
+ # them up above us.
+ node.list = node.next
+ node.next = cur.next
+ cur.next = None
+
+ elif node.list:
+ # The menu node is a choice, menu, or if. Finalize each child node.
+
+ if node.item is MENU:
+ visible_if = self._make_and(visible_if, node.visibility)
+
+ # Propagate the menu node's dependencies to each child menu node.
+ #
+ # This needs to go before the recursive _finalize_node() call so
+ # that implicit submenu creation can look ahead at dependencies.
+ self._propagate_deps(node, visible_if)
+
+ # Finalize the children
+ cur = node.list
+ while cur:
+ self._finalize_node(cur, visible_if)
+ cur = cur.next
+
+ if node.list:
+ # node's children have been individually finalized. Do final steps
+ # to finalize this "level" in the menu tree.
+ _flatten(node.list)
+ _remove_ifs(node)
+
+ # Empty choices (node.list None) are possible, so this needs to go
+ # outside
+ if node.item.__class__ is Choice:
+ # Add the node's non-node-specific properties to the choice, like
+ # _add_props_to_sym() does
+ choice = node.item
+ choice.direct_dep = self._make_or(choice.direct_dep, node.dep)
+ choice.defaults += node.defaults
+
+ _finalize_choice(node)
+
+ def _propagate_deps(self, node, visible_if):
+ # Propagates 'node's dependencies to its child menu nodes
+
+ # If the parent node holds a Choice, we use the Choice itself as the
+ # parent dependency. This makes sense as the value (mode) of the choice
+ # limits the visibility of the contained choice symbols. The C
+ # implementation works the same way.
+ #
+ # Due to the similar interface, Choice works as a drop-in replacement
+ # for Symbol here.
+ basedep = node.item if node.item.__class__ is Choice else node.dep
+
+ cur = node.list
+ while cur:
+ dep = cur.dep = self._make_and(cur.dep, basedep)
+
+ if cur.item.__class__ in _SYMBOL_CHOICE:
+ # Propagate 'visible if' and dependencies to the prompt
+ if cur.prompt:
+ cur.prompt = (cur.prompt[0],
+ self._make_and(
+ cur.prompt[1],
+ self._make_and(visible_if, dep)))
+
+ # Propagate dependencies to defaults
+ if cur.defaults:
+ cur.defaults = [(default, self._make_and(cond, dep))
+ for default, cond in cur.defaults]
+
+ # Propagate dependencies to ranges
+ if cur.ranges:
+ cur.ranges = [(low, high, self._make_and(cond, dep))
+ for low, high, cond in cur.ranges]
+
+ # Propagate dependencies to selects
+ if cur.selects:
+ cur.selects = [(target, self._make_and(cond, dep))
+ for target, cond in cur.selects]
+
+ # Propagate dependencies to implies
+ if cur.implies:
+ cur.implies = [(target, self._make_and(cond, dep))
+ for target, cond in cur.implies]
+
+ elif cur.prompt: # Not a symbol/choice
+ # Propagate dependencies to the prompt. 'visible if' is only
+ # propagated to symbols/choices.
+ cur.prompt = (cur.prompt[0],
+ self._make_and(cur.prompt[1], dep))
+
+ cur = cur.next
+
+ def _add_props_to_sym(self, node):
+ # Copies properties from the menu node 'node' up to its contained
+ # symbol, and adds (weak) reverse dependencies to selected/implied
+ # symbols.
+ #
+ # This can't be rolled into _propagate_deps(), because that function
+ # traverses the menu tree roughly breadth-first, meaning properties on
+ # symbols defined in multiple locations could end up in the wrong
+ # order.
+
+ sym = node.item
+
+ # See the Symbol class docstring
+ sym.direct_dep = self._make_or(sym.direct_dep, node.dep)
+
+ sym.defaults += node.defaults
+ sym.ranges += node.ranges
+ sym.selects += node.selects
+ sym.implies += node.implies
+
+ # Modify the reverse dependencies of the selected symbol
+ for target, cond in node.selects:
+ target.rev_dep = self._make_or(
+ target.rev_dep,
+ self._make_and(sym, cond))
+
+ # Modify the weak reverse dependencies of the implied
+ # symbol
+ for target, cond in node.implies:
+ target.weak_rev_dep = self._make_or(
+ target.weak_rev_dep,
+ self._make_and(sym, cond))
+
+ #
+ # Misc.
+ #
+
+ def _check_sym_sanity(self):
+ # Checks various symbol properties that are handiest to check after
+ # parsing. Only generates errors and warnings.
+
+ def num_ok(sym, type_):
+ # Returns True if the (possibly constant) symbol 'sym' is valid as a value
+ # for a symbol of type type_ (INT or HEX)
+
+ # 'not sym.nodes' implies a constant or undefined symbol, e.g. a plain
+ # "123"
+ if not sym.nodes:
+ return _is_base_n(sym.name, _TYPE_TO_BASE[type_])
+
+ return sym.orig_type is type_
+
+ for sym in self.unique_defined_syms:
+ if sym.orig_type in _BOOL_TRISTATE:
+ # A helper function could be factored out here, but keep it
+ # speedy/straightforward
+
+ for target_sym, _ in sym.selects:
+ if target_sym.orig_type not in _BOOL_TRISTATE_UNKNOWN:
+ self._warn("{} selects the {} symbol {}, which is not "
+ "bool or tristate"
+ .format(sym.name_and_loc,
+ TYPE_TO_STR[target_sym.orig_type],
+ target_sym.name_and_loc))
+
+ for target_sym, _ in sym.implies:
+ if target_sym.orig_type not in _BOOL_TRISTATE_UNKNOWN:
+ self._warn("{} implies the {} symbol {}, which is not "
+ "bool or tristate"
+ .format(sym.name_and_loc,
+ TYPE_TO_STR[target_sym.orig_type],
+ target_sym.name_and_loc))
+
+ elif sym.orig_type: # STRING/INT/HEX
+ for default, _ in sym.defaults:
+ if default.__class__ is not Symbol:
+ raise KconfigError(
+ "the {} symbol {} has a malformed default {} -- "
+ "expected a single symbol"
+ .format(TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc, expr_str(default)))
+
+ if sym.orig_type is STRING:
+ if not default.is_constant and not default.nodes and \
+ not default.name.isupper():
+ # 'default foo' on a string symbol could be either a symbol
+ # reference or someone leaving out the quotes. Guess that
+ # the quotes were left out if 'foo' isn't all-uppercase
+ # (and no symbol named 'foo' exists).
+ self._warn("style: quotes recommended around "
+ "default value for string symbol "
+ + sym.name_and_loc)
+
+ elif not num_ok(default, sym.orig_type): # INT/HEX
+ self._warn("the {0} symbol {1} has a non-{0} default {2}"
+ .format(TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc,
+ default.name_and_loc))
+
+ if sym.selects or sym.implies:
+ self._warn("the {} symbol {} has selects or implies"
+ .format(TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc))
+
+ else: # UNKNOWN
+ self._warn("{} defined without a type"
+ .format(sym.name_and_loc))
+
+
+ if sym.ranges:
+ if sym.orig_type not in _INT_HEX:
+ self._warn(
+ "the {} symbol {} has ranges, but is not int or hex"
+ .format(TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc))
+ else:
+ for low, high, _ in sym.ranges:
+ if not num_ok(low, sym.orig_type) or \
+ not num_ok(high, sym.orig_type):
+
+ self._warn("the {0} symbol {1} has a non-{0} "
+ "range [{2}, {3}]"
+ .format(TYPE_TO_STR[sym.orig_type],
+ sym.name_and_loc,
+ low.name_and_loc,
+ high.name_and_loc))
+
+ def _check_choice_sanity(self):
+ # Checks various choice properties that are handiest to check after
+ # parsing. Only generates errors and warnings.
+
+ def warn_select_imply(sym, expr, expr_type):
+ msg = "the choice symbol {} is {} by the following symbols, but " \
+ "select/imply has no effect on choice symbols" \
+ .format(sym.name_and_loc, expr_type)
+
+ # si = select/imply
+ for si in split_expr(expr, OR):
+ msg += "\n - " + split_expr(si, AND)[0].name_and_loc
+
+ self._warn(msg)
+
+ for choice in self.unique_choices:
+ if choice.orig_type not in _BOOL_TRISTATE:
+ self._warn("{} defined with type {}"
+ .format(choice.name_and_loc,
+ TYPE_TO_STR[choice.orig_type]))
+
+ for node in choice.nodes:
+ if node.prompt:
+ break
+ else:
+ self._warn(choice.name_and_loc + " defined without a prompt")
+
+ for default, _ in choice.defaults:
+ if default.__class__ is not Symbol:
+ raise KconfigError(
+ "{} has a malformed default {}"
+ .format(choice.name_and_loc, expr_str(default)))
+
+ if default.choice is not choice:
+ self._warn("the default selection {} of {} is not "
+ "contained in the choice"
+ .format(default.name_and_loc,
+ choice.name_and_loc))
+
+ for sym in choice.syms:
+ if sym.defaults:
+ self._warn("default on the choice symbol {} will have "
+ "no effect, as defaults do not affect choice "
+ "symbols".format(sym.name_and_loc))
+
+ if sym.rev_dep is not sym.kconfig.n:
+ warn_select_imply(sym, sym.rev_dep, "selected")
+
+ if sym.weak_rev_dep is not sym.kconfig.n:
+ warn_select_imply(sym, sym.weak_rev_dep, "implied")
+
+ for node in sym.nodes:
+ if node.parent.item is choice:
+ if not node.prompt:
+ self._warn("the choice symbol {} has no prompt"
+ .format(sym.name_and_loc))
+
+ elif node.prompt:
+ self._warn("the choice symbol {} is defined with a "
+ "prompt outside the choice"
+ .format(sym.name_and_loc))
+
+ def _parse_error(self, msg):
+ raise KconfigError("{}error: couldn't parse '{}': {}".format(
+ "" if self.filename is None else
+ "{}:{}: ".format(self.filename, self.linenr),
+ self._line.strip(), msg))
+
+ def _trailing_tokens_error(self):
+ self._parse_error("extra tokens at end of line")
+
+ def _open(self, filename, mode):
+ # open() wrapper:
+ #
+ # - Enable universal newlines mode on Python 2 to ease
+ # interoperability between Linux and Windows. It's already the
+ # default on Python 3.
+ #
+ # The "U" flag would currently work for both Python 2 and 3, but it's
+ # deprecated on Python 3, so play it future-safe.
+ #
+ # io.open() defaults to universal newlines on Python 2 (and is an
+ # alias for open() on Python 3), but it returns 'unicode' strings and
+ # slows things down:
+ #
+ # Parsing x86 Kconfigs on Python 2
+ #
+ # with open(..., "rU"):
+ #
+ # real 0m0.930s
+ # user 0m0.905s
+ # sys 0m0.025s
+ #
+ # with io.open():
+ #
+ # real 0m1.069s
+ # user 0m1.040s
+ # sys 0m0.029s
+ #
+ # There's no appreciable performance difference between "r" and
+ # "rU" for parsing performance on Python 2.
+ #
+ # - For Python 3, force the encoding. Forcing the encoding on Python 2
+ # turns strings into Unicode strings, which gets messy. Python 2
+ # doesn't decode regular strings anyway.
+ return open(filename, "rU" if mode == "r" else mode) if _IS_PY2 else \
+ open(filename, mode, encoding=self._encoding)
+
+ def _check_undef_syms(self):
+ # Prints warnings for all references to undefined symbols within the
+ # Kconfig files
+
+ def is_num(s):
+ # Returns True if the string 's' looks like a number.
+ #
+ # Internally, all operands in Kconfig are symbols, only undefined symbols
+ # (which numbers usually are) get their name as their value.
+ #
+ # Only hex numbers that start with 0x/0X are classified as numbers.
+ # Otherwise, symbols whose names happen to contain only the letters A-F
+ # would trigger false positives.
+
+ try:
+ int(s)
+ except ValueError:
+ if not s.startswith(("0x", "0X")):
+ return False
+
+ try:
+ int(s, 16)
+ except ValueError:
+ return False
+
+ return True
+
+ for sym in (self.syms.viewvalues if _IS_PY2 else self.syms.values)():
+ # - sym.nodes empty means the symbol is undefined (has no
+ # definition locations)
+ #
+ # - Due to Kconfig internals, numbers show up as undefined Kconfig
+ # symbols, but shouldn't be flagged
+ #
+ # - The MODULES symbol always exists
+ if not sym.nodes and not is_num(sym.name) and \
+ sym.name != "MODULES":
+
+ msg = "undefined symbol {}:".format(sym.name)
+ for node in self.node_iter():
+ if sym in node.referenced:
+ msg += "\n\n- Referenced at {}:{}:\n\n{}" \
+ .format(node.filename, node.linenr, node)
+ self._warn(msg)
+
+ def _warn(self, msg, filename=None, linenr=None):
+ # For printing general warnings
+
+ if not self.warn:
+ return
+
+ msg = "warning: " + msg
+ if filename is not None:
+ msg = "{}:{}: {}".format(filename, linenr, msg)
+
+ self.warnings.append(msg)
+ if self.warn_to_stderr:
+ sys.stderr.write(msg + "\n")
+
+
+class Symbol(object):
+ """
+ Represents a configuration symbol:
+
+ (menu)config FOO
+ ...
+
+ The following attributes are available. They should be viewed as read-only,
+ and some are implemented through @property magic (but are still efficient
+ to access due to internal caching).
+
+ Note: Prompts, help texts, and locations are stored in the Symbol's
+ MenuNode(s) rather than in the Symbol itself. Check the 