1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
|
# ImageBuilder
Building an embedded virtualized system with anything more than one
Domain can be difficult, error prone and time consuming.
ImageBuilder, an Open Source collection of scripts (contributions
encouraged), changes all that.
ImageBuilder generates a U-Boot script that can be used to load all of
the binaries automatically and boot the full system fast. Given a
collection of binaries such as Xen, Dom0 and a number of Dom0-less
DomUs, ImageBuilder takes care of calculating all loading addresses,
editing device tree with the necessary information, and even
pre-configuring a disk image with kernels and rootfses.
ImageBuilder has been tested on Xilinx ZynqMP MPSoC boards. An
up-to-date wikipage is also available at
[wiki.xenproject.org](https://wiki.xenproject.org/index.php?title=ImageBuilder).
## scripts/uboot-script-gen
The ImageBuilder script that generates a u-boot script to load all your
binaries for a Xen Dom0-less setup is `scripts/uboot-script-gen`.
To use it, first write a config file like `config`:
```
MEMORY_START="0x0"
MEMORY_END="0x80000000"
LOAD_CMD="tftpb"
BOOT_CMD="booti"
DEVICE_TREE="mpsoc.dtb"
XEN="xen"
XEN_CMD="console=dtuart dtuart=serial0 dom0_mem=1G dom0_max_vcpus=1 bootscrub=0 vwfi=native sched=null"
PASSTHROUGH_DTS_REPO="git@github.com:Xilinx/xen-passthrough-device-trees.git device-trees-2021.2/zcu102"
DOM0_KERNEL="Image-dom0"
DOM0_CMD="console=hvc0 earlycon=xen earlyprintk=xen clk_ignore_unused"
DOM0_RAMDISK="dom0-ramdisk.cpio"
DOM0_MEM=1024
DOM0_VCPUS=1
NUM_DT_OVERLAY=1
DT_OVERLAY[0]="host_dt_overlay.dtbo"
NUM_DOMUS=2
DOMU_KERNEL[0]="zynqmp-dom1/Image-domU"
DOMU_PASSTHROUGH_PATHS[0]="/axi/ethernet@ff0e0000 /axi/serial@ff000000"
DOMU_CMD[0]="console=ttyPS0 earlycon console=ttyPS0,115200 clk_ignore_unused rdinit=/sbin/init root=/dev/ram0 init=/bin/sh"
DOMU_RAMDISK[0]="zynqmp-dom1/domU-ramdisk.cpio"
DOMU_COLORS[0]="6-14"
DOMU_KERNEL[1]="zynqmp-dom2/Image-domU"
DOMU_CMD[1]="console=ttyAMA0 clk_ignore_unused rdinit=/sbin/init root=/dev/ram0 init=/bin/sh"
DOMU_RAMDISK[1]="zynqmp-dom2/domU-ramdisk.cpio"
DOMU_MEM[1]=512
DOMU_VCPUS[1]=1
BITSTREAM=download.bit
NUM_BOOT_AUX_FILE=2
BOOT_AUX_FILE[0]="BOOT.BIN"
BOOT_AUX_FILE[1]="uboot.cfg"
UBOOT_SOURCE="boot.source"
UBOOT_SCRIPT="boot.scr"
APPEND_EXTRA_CMDS="extra.txt"
FDTEDIT="imagebuilder.dtb"
FIT="boot.fit"
FIT_ENC_KEY_DIR="dir/key"
FIT_ENC_UB_DTB="uboot.dtb"
```
Where:
- MEMORY_START and MEMORY_END specify the start and end of RAM.
- LOAD_CMD specifies the u-boot command used to load the binaries. This
can be left out of the config and be (over)written by the -t CLI
argument. It has to be set either in the config file or CLI argument
though.
- BOOT_CMD specifies the u-boot command used to boot the binaries.
By default, it is 'booti'. The acceptable values are 'booti', 'bootm'
and 'bootefi' and 'none'. If the value is 'none', the BOOT_CMD is not
added to the boot script, and the addresses for the Xen binary and the
DTB are stored in 'host_kernel_addr' and 'host_fdt_addr' u-boot
env variables respectively, to be used manually when booting.
- DEVICE_TREE specifies the DTB file to load.
- XEN specifies the Xen hypervisor binary to load. Note that it has to
be a regular Xen binary, not a u-boot binary.
- XEN_COLORS specifies the colors (cache coloring) to be used for Xen
and is in the format startcolor-endcolor
- XEN_CMD specifies the command line arguments used for Xen. If not
set, the default one will be used.
- PASSTHROUGH_DTS_REPO specifies the git repository and/or the directory
which contains the partial device trees. This is optional. However, if
this is specified, then DOMU_PASSTHROUGH_PATHS[number] need to be specified.
uboot-script-gen will compile the partial device trees which have
been specified in DOMU_PASSTHROUGH_PATHS[number].
- DOM0_KERNEL specifies the Dom0 kernel file to load.
For dom0less configurations, the parameter is optional.
- DOM0_MEM specifies the amount of memory for Dom0 VM in MB. The default
is 1024. This is only applicable when XEN_CMD is not specified.
- DOM0_VCPUS specifies the number of VCPUs for Dom0. The default is 1. This is
only applicable when XEN_CMD is not specified.
- DOM0_COLORS specifies the colors (cache coloring) to be used for dom0
and is in the format startcolor-endcolor
- DOM0_CMD specifies the command line arguments for Dom0's Linux
kernel. If "root=" isn't set, imagebuilder will try to determine it.
If not set at all, the default one is used.
- DOM0_RAMDISK specifies the Dom0 ramdisk to use. Note that it should be
a regular ramdisk cpio.gz file, not a u-boot binary.
- NUM_DT_OVERLAY specifies the number of host device tree overlays to be
added at boot time in u-boot
- DT_OVERLAY[number] specifies the path to the hosts device tree overlays
to be added at boot time in u-boot
- NUM_DOMUS specifies how many Dom0-less DomUs to load
- DOMU_KERNEL[number] specifies the DomU kernel to use.
- DOMU_CMD[number] specifies the command line arguments for DomU's Linux
kernel. If not set, then "console=ttyAMA0" is used.
- DOMU_RAMDISK[number] specifies the DomU ramdisk to use.
- DOMU_PASSTHROUGH_PATHS[number] specifies the passthrough devices (
separated by spaces). It adds "xen,passthrough" to the corresponding
dtb nodes in xen device tree blob.
This option is valid in the following two cases:
1. When PASSTHROUGH_DTS_REPO is provided.
With this option, the partial device trees (corresponding to the
passthrough devices) from the PASSTHROUGH_DTS_REPO, are compiled
merged and used as DOMU[number] device tree blob.
Note it assumes that the names of the partial device trees will match
to the names of the devices specified here.
2. When DOMU_NOBOOT[number] is provided. In this case, it will only
add "xen,passthrough" as mentioned before.
- DOMU_PASSTHROUGH_DTB[number] specifies the passthrough device trees
blob. This option is used when DOMU_PASSTHROUGH_PATHS[number] is not
specified by the user.
NOTE that with this option, user needs to manually set xen,passthrough
in xen.dtb.
- DOMU_MEM[number] is the amount of memory for the VM in MB, default 512MB
- DOMU_VCPUS[number] is the number of vcpus for the VM, default 1
- DOMU_COLORS[number] specifies the colors (cache coloring) to be used
for the domain and is in the format startcolor-endcolor
- DOMU_NOBOOT[number]: if specified, the DomU is not started
automatically at boot as dom0-less guest. It can still be created
later from Dom0.
- DOMU_STATIC_MEM[number]="baseaddr1 size1 ... baseaddrN sizeN"
if specified, indicates the host physical address regions
[baseaddr, baseaddr + size) to be reserved to the VM for static allocation.
- DOMU_DIRECT_MAP[number] can be set to 1 or 0.
If set to 1, the VM is direct mapped. The default is 1.
This is only applicable when DOMU_STATIC_MEM is specified.
- DOMU_ENHANCED[number] can be set to 1 or 0, default is 1 when Dom0 is
present. If set to 1, the VM can use PV drivers. Older Linux kernels
might break.
- DOMU_CPUPOOL[number] specifies the id of the cpupool (created using
CPUPOOL[number] option, where number == id) that will be assigned to domU.
- LINUX is optional but specifies the Linux kernel for when Xen is NOT
used. To enable this set any LINUX\_\* variables and do NOT set the
XEN variable.
- LINUX_CMD is optional but specifies the baremetal Linux kernel CMD.
- LINUX_RAMDISK is optional but specifies the ramdisk to use when
booting baremetal Linux.
- BITSTREAM is optional but specifies the bitstream to program the FPGA
with in u-boot when booting. Currently only a single bitstream is
supported.
- NUM_BOOT_AUX_FILE: is optional but if specified tell how many extra
files to include in the first partition with disk_image. Useful for
things like uboot config files, firmware files or other such files.
- BOOT_AUX_FILE[number]: a list of file(s) to be included in the first
partition.
- UBOOT_SOURCE and UBOOT_SCRIPT specify the output. They are optional
as you can pass -o FILENAME to uboot-script-gen as a command line
parameter
- APPEND_EXTRA_CMDS: is optional and specifies the path to a text file
containing extra u-boot commands to be added to the boot script before
the boot command. Useful for running custom fixup commands.
- FDTEDIT is an optional and is off by default. Specifies the output
modified dtb, used for reference and fdt_std.
- FIT is an optional and is off by default. Specifies using a fit image
for booting rather than individual files.
- FIT_ENC_KEY_DIR is optional but specifies the directory and hint used
for signing a FIT image. The CLI arguments overwrite what's in the
config. See the -u option below for more information.
- FIT_ENC_UB_DTB is optional but specifies the u-boot dtb to modify and
include the public key in. This can only be used with
FIT_ENC_KEY_DIR. See the -u option below for more information.
- CPUPOOL[number]="cpu@1,...,cpu@N scheduler"
specifies the list of cpus' node names (separated by commas) and the scheduler
to be used to create boot-time cpupool. If no scheduler is set, the Xen
default one will be used.
- NUM_CPUPOOLS specifies the number of boot-time cpupools to create.
Then you can invoke uboot-script-gen as follows:
```
$ bash ./scripts/uboot-script-gen -c /path/to/config-file -d . -t tftp -o bootscript
```
Where:\
-c specifies the path to the config file to use\
-d specifies the "root" directory (paths in the config file are relative
to it), this is not a working directory (any output file locations
are specified in the config and any temporary files are in /tmp)\
-t specifies the u-boot command to load the binaries. "tftp", "sd", "usb"
and "scsi" are shorthands for "tftpb", "load mmc 0:1", "fatload usb 0:1"
and "load scsi 0:1", but actually any arbitrary command can be used,
for instance -t "fatload" is valid. The only special commands are:
fit, which generates a FIT image using a script, and fit_std, which
produces a standard style of fit image without a script, but has
issues with dom0less configurations and isn't recommended. \
-o specifies the output filename for the uboot script and its source.\
-k specifies the key directory for signing images in a FIT image and the
hint. The hint is the name of the crt and key files minus the
suffix (<hint>.key, <hint>.crt). This is optional and but enables
signature for the fit or fit_std -t options.\
-u specifies the U-boot control dtb. This is an optional argument but
can only be used in combination with the -k option. This adds the
public key into the dtb. Then one can add this dtb back into the
u-boot bin or elf.\
### Signed FIT images
Signed FIT images are a way to sign images with asymmetrical keys. While
making the FIT image, images are signed with a private key; then during
boot U-Boot uses a public key in its control dtb to verify the
signatures. Some of the U-Boot config options needed are:
CONFIG_FIT_SIGNATURE=y\
CONFIG_RSA=y\
CONFIG_LEGACY_IMAGE_FORMAT=n\
Once U-boot is built, then take the control dtb, supply it to
Imagebuilder when building a signed image, then use it when booting.
For generating the keys and other documentation, see:\
u-boot/doc/uImage.FIT/signature.txt\
## scripts/disk\_image
The ImageBuilder script that generates a disk image file to load on a SD
or SATA drive. This creates multiple partitions: 1 boot partition where
the boot files from working directory (-c option) are, and 1 additional
partition for each Dom0/DomU cpio archive to write to disk.
disk\_image will write to disk as separate partition each file specified
as follows:
- DOM0_ROOTFS specifies the Dom0 rootfs to use.
- DOMU_ROOTFS[number] specifies the DomU rootfs to use.
The provided rootfs should not be a u-boot binary. The supported
formats are:
- cpio.gz
- tar.gz
After you've generated the u-boot scripts using the uboot-script-gen
script, disk_image is run as follows:
```
$ sudo bash ./scripts/disk_image -c /path/to/config-file -d . \
-w /path/to/tmp/dir \
-o /path/to/output/disk.img \
-t "sd"
```
Where:\
-c specifies the path to the config file to use\
-d specifies the working directory (paths in the config file are relative
to it)\
-w specifies the temporary working directory that the script uses for
building the disk image, and if not set, one is created in /tmp\
-o specifies the output disk image file name\
-a specifies whether the disk image size is to be aligned to the nearest
power of two\
-t specifies the u-boot command to load the binaries. "tftp", "sd", "usb"
and "scsi" are shorthands for "tftpb", "load mmc 0:1", "fatload usb 0:1"
and "load scsi 0:1", but actually any arbitrary command can be used,
for instance -t "fatload" is valid.
disk\_image supports these additional parameters on the config file:
- GPT is optional and select the usage of a GPT partition table (sgdisk
is required)
disk_image also generates on the fly a xl config file for each domU and
adds them to the dom0 rootfs partition under /etc/xen. It makes it
easier to start those domUs from dom0.
|