-
Notifications
You must be signed in to change notification settings - Fork 7
/
cckddasd.html
745 lines (718 loc) · 35.3 KB
/
cckddasd.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 3.0//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML>
<HEAD><TITLE>
Hercules: Compressed Dasd Emulation</TITLE>
<LINK REL=STYLESHEET TYPE="text/css" HREF="hercules.css">
</HEAD>
<BODY BGCOLOR="#ffffcc" TEXT="#000000" LINK="#0000A0"
VLINK="#008040" ALINK="#000000">
<h1>Compressed DASD Emulation</h1>
<hr noshade>
<h2>Contents</h2>
<ul>
<li><a href="#introduction"> Introduction </a>
<li><a href="#shadowfiles"> Shadow Files </a>
<li><a href="#filestructure"> File Structure </a>
<li><a href="#howitworks"> How It Works </a>
<li><a href="#cckdcommand"> The CCKD Command (and initialization statement) </a>
<li><a href="#utilities"> Utilities </a>
</ul>
<hr noshade>
<h2><a NAME="introduction">Introduction</a></h2>
<p>
Using compressed DASD files you can significantly reduce the file space
required for emulated DASD files and possibly gain a performance boost
because less physical I/O occurs.
Both <b>CKD</b> (Count-Key-Data) and <b>FBA</b> (Fixed-Block-Architecture)
emulation files can be compressed.
<p>
In regular (or uncompressed) files, each CKD track or FBA block occupies
a specific spot in the emulation file. The offset of the track or block
in the file can be directly calculated knowing the track or block number
and the maximum size of the track or block. In compressed files, each
track image or group of blocks may be compressed by
<a href="http://www.zlib.net/"><b>zlib</b></a> or
<a href="http://www.bzip.org/"><b>bzip2</b></a>, and only
occupies the space neccessary for the compressed image. The offset of a compressed
track or block is obtained by performing a two-table lookup. The lookup
tables themselves reside in the emulation file.
<p>
Because FBA blocks are 512 bytes in length, and that being a rather small
number, FBA blocks are grouped into <b>block groups</b>. Each block group
contains 120 FBA blocks (60K).
<p>
Whenever a track or block group is written to a compressed file,
it is written either to an existing free space within the file, or at
the end of the file, then the lookup tables are updated, and then the space the
track or block group previously occupied is freed. The location of a
track or block group in the file can change many times.
<p>
In the event of a failure (for example, Hercules crash,
operating system crash or power failure), the compressed emulation file
on the host's physical disk may be out of sync if the host operating
system defers physical writes to the file system containing the emulation
file. Several methods have evolved to reduce the amount of data lost in
these kind of events.
<p>
A compressed file may occupy only 20% of the disk space required by an
uncompressed file. In other words, you may be able to have 5 times more
emulated volumes using compressed DASD files. However, compressed files
are more sensitive to failures and corruption may occur.
<p>
<hr noshade>
<p><h2><a NAME="shadowfiles">Shadow Files</a></h2>
<p>
An compressed CKD or FBA dasd can have more than one physical file. The
additional files are called <em>shadow files</em>.
The function is implemented as a kind of
<i>snapshot</i>, where a new shadow file can be created on demand.
An emulated dasd is represented by a <em>base</em> file and 0 or more
shadow files. All files are opened <em>read-only</em>
except for the <em>current</em> file, which is opened <em>read-write</em>.
<p>
Shadow files are specified by the <b>sf=</b><i>shadow-file-name</i> parameter
on the device statement for the compressed DASD device.
<p class="note">
Please note that the specified shadow filename does <u>not</u> have to
actually exist. The <i>shadow-file-name</i> operand of the <i>sf=</i>
parameter is simply a filename <i><u>template</u></i> that will be used
to name the shadow file whenever a shadow file is to be created, but
shadow files do not actually get created until you specifically create
them via the <code>sf+xxxx</code> (or <code>sf+*</code>) command. Please
refer to the discussion of the <b>sf</b> command several paragraphs
below for more information.
<p>
The shadow file name
should have spot where the shadow file number will be set. This is
either the character preceding the last period after the last slash or the
last character if there is no period. For example:<br><br>
<code>0100 3390 disks/linux1.dsk sf=shadows/linux1_*.dsk</code>
<p>
There can be up to 8 shadow files in use at any time for an
emulated dasd device. The base file is designated file<b>[0]</b> and
the shadow files are file<b>[1]</b> to file<b>[8]</b>.
The <em>highest</em> numbered file in use at a given time is the <em>current</em>
file, where all writes will occur. Track reads start with the <em>current</em>
file and proceed down until a file is found that actually contains the track
image.
<p>
A shadow file contains all the changes made to the emulated dasd
since it was created, until the next shadow file is created. The moment
of the shadow file's creation can be thought of as a <em>snapshot</em>
of the current emulated dasd at that time, because if the shadow file is
later removed, then the emulated dasd reverts back to the state it was at
when the <em>snapshot</em> was taken.
<p>
Using shadow files, you can keep the base file on a read-only device
such as cdrom, or change the base file attributes to read-only,
ensuring that this file can never be corrupted.
<p>
Hercules console commands are provided to add a new shadow file, remove
the current shadow file (with or without backward merge), compress the
curent shadow file, and display the shadow file status and statistics:<br><br>
<table>
<tr><td align="left"><b>sf+</b></td>
<td align="left" colspan="2"><font size=-1>unit</font></td>
<td align="left">Create a new shadow file</td>
<tr><td align="left" valign="top"><b>sf-</b></td>
<td align="left" valign="top"><font size=-1>unit</font></td>
<td align="left" valign="top"<font size=-1><b>merge</b><br><em>nomerge<br>force</em></font></td>
<td align="left" valign="top">Remove a shadow file. If <em>merge</em> is
specified or defaulted, then the contents of the current
file is merged into the previous file, the current file
is removed, and the previous file becomes the current file.
The previous file must be able to be opened read-write.
If <em>nomerge</em> is specified then the contents of the
current file is discarded and the previous file becomes the
current file. However, if the previous file is read-only,
then a new shadow file is created (re-added) and that becomes
the current file.
The <em>force</em> option is required when doing a merge to
the base file and the base file is read-only because the
<em>ro</em> option was specified on the device config statement.
</td>
<tr><td align="left"><b>sfc</b></td>
<td align="left" colspan="2"><font size=-1>unit</font></td>
<td align="left">Compress the current file</td>
<tr><td align="left" valign="top"><b>sfk</b></td>
<td align="left" valign="top"><font size=-1>unit</font></td>
<td align="left" valign="top"<font size=-1><i>level</i></font></td>
<td align="left" valign="top">Perform the <b>chkdsk</b> function on the current file.
Level is a number -1 ... 4, the default is <b>2</b>. The levels are:<br>
-1     devhdr, cdevhdr, l1 table <br>
  0     devhdr, cdevhdr, l1 table, l2 tables <br>
  1     devhdr, cdevhdr, l1 table, l2 tables, free spaces <br>
  2     devhdr, cdevhdr, l1 table, l2 tables, free spaces, trkhdrs <br>
  3     devhdr, cdevhdr, l1 table, l2 tables, free spaces, trkimgs <br>
  4     devhdr, cdevhdr. Build everything else from recovery
</td>
<tr><td align="left"><b>sfd</b></td>
<td align="left" colspan="2"><font size=-1>unit</font></td>
<td align="left">Display shadow file status and statistics</td>
</table>
<br>
<b><font size=-1>Note</font></b>. You can use <b>*</b> in place of unit
address to apply the command to all compressed dasd (e.g. <code>'sf+*'</code>,
or <code>'sf-* nomerge'</code>).
<p>
<hr noshade>
<p><h2><a NAME="filestructure">Compressed DASD File Structure</a></h2>
<p>
A compressed DASD file has 6 types of spaces, a <em>device header</em>,
a <em>compressed device header</em>, a <em>primary lookup table</em>,
<em>secondary lookup tables</em>, track or block group <em>images</em>,
and <em>free spaces</em>. The first 3 types only occur once, at the
beginning of the file in order. The rest of the file is occupied by
the other 3 space types.
<p>
The first 512 bytes of a compressed DASD file contains a <b>device header</b>.
The device header contains an eye-catcher that identifies the file type
(CKD or FBA and base or shadow). The device type and file size is also
specified in this header. The header is identical to the header used
for uncompressed CKD files, except for the eye-catcher:
<p>
<table border=1>
<tr><td align="left" colspan="8"><font size=-1>devid</font></td>
<td align="left" colspan="4"><font size=-1>heads</font></td>
<td align="left" colspan="4"><font size=-1>trksize</font></td>
<tr><td align="left" colspan="1"><font size=-1>devt</font></td>
<td align="left" colspan="1"><font size=-1>seq</font></td>
<td align="left" colspan="2"><font size=-1>hicyl</font></td>
<td align="left" colspan="12"> </td>
<tr><td align="center" valign="middle" colspan="16">
<br><br><font size=-1>reserved</font><br><br><br></td>
</table>
<p>
The next 512 bytes contains the <b>compressed device header</b>.
This contains file usage information such as the amount of free
space in the file:
<p>
<table border=1>
<tr><td align="left" colspan="3"><font size=-1>vrm</font></td>
<td align="left" colspan="1"><font size=-1>opts</font></td>
<td align="left" colspan="4"><font size=-1>numl1</font></td>
<td align="left" colspan="4"><font size=-1>numl2</font></td>
<td align="left" colspan="4"><font size=-1>size</font></td>
<tr><td align="left" colspan="4"><font size=-1>used</font></td>
<td align="left" colspan="4"><font size=-1>->free</font></td>
<td align="left" colspan="4"><font size=-1>free</font></td>
<td align="left" colspan="4"><font size=-1>largest</font></td>
<tr><td align="left" colspan="4"><font size=-1>number</font></td>
<td align="left" colspan="4"><font size=-1> </font></td>
<td align="left" colspan="4"><font size=-1>cyls</font></td>
<td align="left" colspan="1"><font size=-1> </font></td>
<td align="left" colspan="1"><font size=-1>comp</font></td>
<td align="left" colspan="4"><font size=-1>parm</font></td>
<tr><td align="center" colspan="16">
<br><br><font size=-1>reserved</font><br><br><br></td>
</table>
<p>
After the compressed device header is the <b>primary lookup table</b>,
also called the <em>level 1 table</em> or <em>l1tab</em>. Each
4 byte unsigned entry in the l1tab contains the file offset of
a <em>secondary lookup table</em> or <em>level 2 table</em> or
<em>l2tab</em>. The track or block group number being accessed
divided by 256 gives the index into the l1tab. That is, each l1tab
entry represents 256 tracks or block groups. The number of entries
in the l1tab is dependent on the size of the emulated device:
<p>
<table border=1>
<tr><td align="left" colspan="4"><font size=-1>l2<sub>0</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>1</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>2</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>3</sub></font></td>
<tr><td align="left" colspan="4"><font size=-1>l2<sub>4</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>5</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>6</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>7</sub></font></td>
<tr><td align="left" colspan="16">
<br><br><center>.     .     .</center><br><br></td>
<tr><td align="left" colspan="4"><font size=-1>l2<sub>n-4</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>n-3</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>n-2</sub></font></td>
<td align="left" colspan="4"><font size=-1>l2<sub>n-1</sub></font></td>
</table>
<p>
Following the <em>l1tab</em>,
in no particular order, are <em>l2tabs</em>, track or block group
<em>images</em>, and <em>free spaces</em>.
<p>
Each <b>secondary lookup table</b> (or <em>l2tab</em>), contains 256 8-byte
entries. The entry is indexed
by the remainder of the track or block group number divided by 256. Each
entry contains an unsigned 4 byte offset, an unsigned 2 byte length and an
unsigned 2 byte size. The length is the space required for the track or
block group image and the size is the amount of space actually used. The
size may be greater than the length to prevent short free spaces from
accumulating in the file.
<p>
<table border=1>
<tr><td align="left" colspan="4"><font size=-1>
<sup>0</sup>  ->image
       </font></td>
<td align="left" colspan="2"><font size=-1>length</font></td>
<td align="left" colspan="2"><font size=-1>size</font></td>
<tr><td align="left" colspan="4"><font size=-1>
<sup>1</sup>  ->image
       </font></td>
<td align="left" colspan="2"><font size=-1>length</font></td>
<td align="left" colspan="2"><font size=-1>size</font></td>
<tr><td align="center" colspan="8"><font size=-1>
<br>.     .    .<br><br></td>
<tr><td align="left" colspan="4"><font size=-1>
<sup>255</sup>  ->image
       </font></td>
<td align="left" colspan="2"><font size=-1>length</font></td>
<td align="left" colspan="2"><font size=-1>size</font></td>
</table>
<p>
A track or block group <b>image</b> contains two fields, a 5-byte
<em>header</em> and a variable amount of data that may or may not be
compressed. The length in the l2tab entry includes the length of the
header and the data.
<p>
<table border=1>
<tr><td align="left"><font size=-1>hdr</font></td>
<td align="left"><font size=-1>track or block group data</font></td>
</table>
<p>
The 5 byte header contains a 1 byte compression indicator and 4 bytes that
identify the track or block group. The format of the identifier
depends on whether the emulated device is CKD or FBA:
<p>
CKD hdr
<table border=1>
<tr><td><font size=-1>comp</font></td>
<td align="left" colspan="2"><font size=-1><b>CC</b></font>  </td>
<td align="left" colspan="2"><font size=-1><b>HH</b></font>  </td>
</table>
<p>
The 2 byte CC is the cylinder number for the track image and the HH
is the head number. These numbers are stored in <em>big-endian</em>
byte order. When the compression indicator byte is zeroed, the 5 byte header
is identical to the <em>Home Address</em> (or <em>HA</em>) for the track image.
The data, which may or may not be compressed, begins with the <em>R0</em>
count and ends with the <em>end-of-track</em> (or <em>eot</em>) marker,
which is a count field containing <code>FFFFFFFFFFFFFFFF</code>.
The <em>HA</em> plus the uncompressed track data comprise the track image.
<p>
FBA hdr
<table border=1>
<tr><td><font size=-1>comp</font></td>
<td align="left" colspan="4"><font size=-1>nnnn</font>
       </td>
</table>
<p>
The 4 byte nnnn field is the FBA block group number in
<em>big-endian</em> byte order. The data contains 120 FBA blocks,
which may or may not be compressed. Uncompressed, the FBA block
group is 60K. The header for FBA, unlike CKD, is not used as part
of the uncompressed image.
<p>
The compression indicator byte contains the value 0, 1 or 2. Any
other value is invalid.
<table border="1">
<tr><td><center>0</center></td><td>    Data is uncompressed</td>
<tr><td><center>1</center></td><td>    Data is compressed using zlib</td>
<tr><td><center>2</center></td><td>    Data is compressed using bzip2</td>
<tr><td>3 .. 255</td><td>    Not valid</td>
</table>
<p>
<b>Free space</b> contains a 4-byte <i>offset</i> to the next free space, a
4-byte <i>length</i> of the free space, and zero or more bytes of residual data:
<p>
<table border=1>
<tr><td><font size=-1>->next</font></td>
<td><font size=-1>length</font></td>
<td>   <font size=-1>residual</font>   </td>
</table>
<p>
The minimum length of a free space is 8 bytes.
The free space chain is ordered by file offset and no two free spaces are
adjacent. The <em>compressed device header</em> contains the offset to the
first free space. The chain is terminated when a free space has zero offset
to the next free space. The free space chain is read when the file is opened
for read-write and written when the file is closed; while the file is opened,
the free space chain is maintained in storage.
<p>
<hr noshade>
<p><h2><a NAME="howitworks">How It Works</a></h2>
<h3>Reading</h3>
<p>
A track or block group image is read while executing a channel
program or by the <em>readahead</em> thread. An image has to
be read before it is updated or written to. An image may be <em>cached</em>.
If an image is cached, then the channel program may complete
<em>synchronously</em>. This means that if all the data a channel program
accesses is cached and Hercules does not have to perform physical I/O,
then the channel program runs synchronously within the SSCH or SIO
instruction in the <em>CPU</em> thread. All DASD channel programs are started
synchronously. If a CCW in the channel program requires physical I/O
then the channel program is interrupted and restarted at that CCW
<em>asynchronously</em> in a <em>device I/O</em> thread.
<p>
All compressed devices share a common cache; the devices can be a mixture
of FBA and/or CKD device types. Each cache entry contains a pointer to
a 64K buffer containing an uncompressed track or block group image.
If the track or block group image being read is not found in the cache,
then the oldest (or <em>least recently used</em> or <em>LRU</em>) entry that
is not <em>busy</em> is <em>stolen</em>. A cache entry is busy if it is
being read, or last accessed by an <em>active</em> channel program, or updated
but not yet written, or being written. If no cache entries are available then
the read must enter a <em>cache wait</em>. When images are detected to be
accessed sequentially then the readahead thread(s) may be signalled to read
following sequential images.
<p>
<h3>Writing</h3>
<p>
When a cache entry is updated or written to, a bit is turned on indicating
the cache entry has been updated. When a <em>cache wait</em> occurs, or
(more likely) during space recovery, a cache <em>flush</em> is performed.
When the cache is flushed, if any entries have the updated bit on, then
the writer thread(s) are signalled. The writer thread selects the oldest
cache entry with the updated bit on, compresses the image, and writes it
to the file. The new image is written to a new space in the file and then
the space previously occupied by the image is freed. In certain circumstances,
the image may be written under <em>stress</em>. A stress write occurs when
a reading thread is in a <em>cache wait</em> or when a high percentage of
cache entries are pending write. In this circumstance, the compression
parameters are relaxed to reduce the CPU requirements. An image written
under stress is likely to take up more space than the same image written
not under stress. The writer thread(s) run 1 nicer than the CPU thread(s);
compression is a CPU intensive activity.
<p>
<h3>Space Recovery</h3>
<p>
Space recovery is also called, somewhat inaccurately, garbage collection.
The primary function of the space recovery thread, or garbage collector,
is to keep the emulated compressed DASD files as small as possible.
After all, that is the reason for using compressed DASD files in the first
place.
<p>
When a track or block group image is written, it is written to a new
location in the file. It is either written to an existing free space
within the file or to the end of the file, increasing the size of the
file. The space previously occupied by the image is freed, but it is
not immediately available for space allocation requests. Instead, it
is <i>pending</i> free space. It is assigned a <i>pending value</i>
(typically 2) that is decremented each space recovery cycle (typically
every 10 seconds). When the pending value reaches 0 then the space is
available for allocation. This increases the chance that a track or
block group image can be recovered in the event of a failure.
<p>
The space recovery routine relocates track or block group
images towards the beginning of the file, causing free space to move
towards the end of the file. When a free space reaches the end of the
file, it `falls off', that is, the file size is reduced.
<p>
Simply, the space recovery routine selects a space after a sufficiently
large <i>non-pending</i> free space. It then reads and writes consecutive
spaces using the normal cckd read and write routines. The space read will
become <i>pending</i> free space and will hopefully be written to a
<i>non-pending</i> free space occurring earlier in the file. Sometimes it
is necessary to write the space later in the file to increase free space
size earlier in the file. Left to itself, the space recovery routine will
eventually remove all free space from the file. However, it is not
intended to be a replacement for the cckdcomp utility; rather, the intent
is to provide sufficient free space to prevent excessive file growth.
<p>
Another function performed by space recovery is to relocate <em>L2</em> (secondary
lookup) tables towards the beginning of the file. This enables the
<i>chkdsk</i> function to complete more quickly during initialization and
simplifies chkdsk recovery.
<hr noshade>
<p><h2><a name=cckdcommand>The cckd command and initialization statement</a></h2>
<p>
The <b>cckd</b> command and initialization statement can be used to
affect cckd processing. The <b>CCKD</b> initialization statement is specified
as a Hercules configuration file statement and supports the same options as
the <b>cckd</b> command explained below.
<p>
<b>Syntax:</b>
<table>
<tr><td><b>cckd</b></td><td><b>help</b></td><td>Display cckd help</td>
<tr><td><b>cckd</b></td><td><b>stats</b></td>
<td>Display current cckd statistics</td>
<tr><td><b>cckd</b></td><td><b>opts</b></td><td>Display current cckd options</td>
<tr><td><b>cckd</b></td><td>opt=value</td><td>Set a cckd option</td>
<tr><td> </td><td> </td><td>Multiple options may be specified,
separated by a comma with no intervening
blanks.</td>
<tr><td> </td><td><b>comp=</b>n</td><td>Compression to be used</td>
<tr><td> </td><td><b>compparm=</b>n</td><td>Compression parameter to be used</td>
<tr><td> </td><td><b>ra=</b>n</td><td>Number readahead threads</td>
<tr><td> </td><td><b>raq=</b>n</td><td>Readahead queue size</td>
<tr><td> </td><td><b>rat=</b>n</td><td>Number of tracks to readahead</td>
<tr><td> </td><td><b>wr=</b>n</td><td>Number writer threads</td>
<tr><td> </td><td><b>gcint=</b>n</td><td>Garbage collection interval</td>
<tr><td> </td><td><b>gcparm=</b>n</td><td>Garbage collection parameter</td>
<tr><td> </td><td><b>nostress=</b>n</td><td>Turn stress writes on or off</td>
<tr><td> </td><td><b>freepend=</b>n</td><td>Set the free pending value</td>
<tr><td> </td><td><b>fsync=</b>n</td><td>Turn fsync on or off</td>
<tr><td> </td><td><b>trace=</b>n</td><td>Number of trace table entries</td>
<tr><td> </td><td><b>linuxnull=</b>n</td><td>Check for null linux tracks</td>
<tr><td> </td><td><b>gcstart=</b>n</td><td>Start garbage collector</td>
</table>
<p>
<b>Options:</b>
<table>
<tr><td valign="top"><b>comp=</b>n</td>
<td>Compression type:<br>
<b>-1</b> Default<br>
<b>  0</b> None<br>
<b>  1</b> zlib<br>
<b>  2</b> bzip2
<p>
Override the compression used for all cckd files. -1 (default) means
don't override the compression.
<p>
</td>
<tr><td valign="top"><b>compparm=</b>n</td>
<td>Compression parameter. A value between -1 and 9. -1 means use the default
parameter. A higher value generally means more compression at the expense
of cpu and/or storage.
<p>
</td>
<tr><td valign="top"><b>ra=</b>n</td>
<td>Number of readahead threads. When sequential track or block group
access is detected, some number (<em>rat=</em>) of tracks or
block groups are queued (<em>raq=</em>) to be read by one of the
readahead threads.
<p>
The default is <b>2</b>.
<p>
You can specify a number between <b>1</b> and <b>9</b>.
<p>
</td>
<tr><td valign="top"><b>raq=</b>n</td>
<td>Size of the readahead queue. When sequential track or block group
access is detected, some number (<em>rat= </em>) of tracks or
block groups are queued in the readahead queue.
<p>
The default is <b>4</b>.
<p>
You can specify a number between <b>0</b> and <b>16</b> (a value
of zero disables readahead).
<p>
</td>
<tr><td valign="top"><b>rat=</b>n</td>
<td>Number of tracks or block groups to read ahead when sequential access
has been detected.
<p>
The default is <b>2</b>.
<p>
You can specify a number between <b>0</b> and <b>16</b> (a value
of zero disables readahead).
<p>
</td>
<tr><td valign="top"><b>wr=</b>n</td>
<td>Number of writer threads. When the cache is <em>flushed</em> updated
cache entries are marked write pending and a writer thread is signalled.
The writer thread compresses the track or block group and writes the
compressed image to the emulation file. A writer thread is cpu-intensive
while compressing the track or block group and i/o-intensive while writing
the compressed image. The writer thread runs one <em>nicer</em> than
the CPU thread(s).
<p>
The default is <b>2</b>.
<p>
You can specify a number between <b>1</b> and <b>9</b>.
<p>
</td>
<tr><td valign="top"><b>gcint=</b>n</td>
<td>Number of seconds the garbage collector thread waits durinng an interval.
At the end of an interval, the garbage collector performs space recovery,
flushes the cache, and optionally <em>fsync</em>s the emulation file.
(However, the file will not be <em>fsync</em>ed unless at least 5
seconds have elapsed since the last <em>fsync</em>).
<p>
The default is <b>10</b> seconds.
<p>
You can specify a number between <b>1</b> and <b>60</b>.
<p>
</td>
<tr><td valign="top"><b>gcparm=</b>n</td>
<td>A value affecting the amount of data moved during the garbage collector's
space recovery routine. The garbage collector determines an amount of
space to move based on the ratio of free space to used space in an
emulation file, and on the number of free spaces in the file. (The
garbage collector wants to reduce the free space to used space ratio
and the number of free spaces). The value is logarithmic; a value
of 8 means moving 2<sup>8</sup> the selected value while a negative
value similarly decreases the amount to be moved. Normally, 256K
will be moved for a file in an interval. Specifying a value of 8 can
increase the amount to 64M. At least 64K will be moved. Interestingly,
specifying a large value (such as 8) may not increase the garbage
collection efficiency correspondingly.
<p>
The default is <b>0</b>.
<p>
You can specify a number between <b>-8</b> and <b>8</b>.
<p>
<tr><td valign="top"><b>nostress=</b>n </td>
<td>Indicates whether <em>stress</em> writes will occur or not. A track
or block group may be written under stress when a high percentage of
the cache is pending write or when a device i/o thread is waiting for
a cache entry. When a stressed write occurs, the compression algorithm
and/or compression parm may be relaxed, resulting in faster compression
but usually a larger compressed image. If <em>nostress</em> is set
to one, then a stressed situation is ignored. You would typically
set this value to one when you want create the smallest emulation file
possible in exchange for a possible performance degradation.
<p>
The default is <b>0</b>.
<p>
You can specify <b>0</b> (enable stressed writes) or <b>1</b>
(disable stressed writes).
<p>
</td>
<tr><td valign="top"><b>freepend=</b>n </td>
<td>Specifies the <em>free pending</em> value for freed space. When a
track or block group image is written the space it previously occupied
is freed. This space will not be available for future
allocations until <em>n</em> garbage collection intervals have completed.
In the event of a catastrophic failure, previously written track or
block group images should be recoverable if the current image has
not yet been written to the physical disk. By default the value
is set to <b>-1</b>. This means that if <em>fsync</em> is specified
then the value is 1 otherwise it is 2. If 0 is specified then freed
space is immediately available for new allocations.
<p>
The default is <b>-1</b>.
<p>
You can specify a number between <b>-1</b> and <b>4</b>.
<p>
</td>
<tr><td valign="top"><b>fsync=</b>n </td>
<td>Enables or disables <em>fsync</em>. When fsync is enabled, then
the disk emulation file is synchronized with the physical hard
disk at the end of a garbage collection interval (however, no more
often than 5 seconds). This means that if <em>freepend</em> is
non-zero then if a catastrophic error occurs then the emulated disks
<em>should</em> be recovered coherently. However, fsync may cause
performance degradation depending on the host operating system and/or
the host operating system level.
<p>
The default is <b>0</b> (fsync disabled).
<p>
You can specify <b>0</b> (disable fsync) or <b>1</b> (enable fsync).
<p>
</td>
<tr><td valign="top"><b>trace=</b>n </td>
<td>Number of cckd trace entries. You would normally specify a non-zero
value when debugging or capturing a problem in cckd code. When the
problem occurs, you should enter the <b>k</b> Hercules console command
which will print the trace table entries.
<p>
The default is <b>0</b>.
<p>
You can specify a number between <b>0</b> and <b>200000</b>.
Each entry represents 128 bytes. Normally, for debugging, I use
100000.
<p>
</td>
<tr><td valign="top"><b>linuxnull=</b>n </td>
<td>If set to 1 then tracks written to 3390 cckd volumes that were
initialized with the <i>-linux</i> option will be checked if they
are null (that is, if all 12 4096 byte user records contain zeroes).
This is used by the dasdcopy utility.
<p>
The default is <b>0</b>.
<p>
</td>
<tr><td valign="top"><b>gcstart=</b>n </td>
<td>If set to 1 then space recovery will become active on any emulated
disks that have free space. Normally space recovery will ignore emulated
disks until they have been updated.
<p>
The default is <b>0</b>.
<p>
</td>
</table>
<p>
<hr noshade>
<p><h2><a NAME="utilities">Utilities</a></h2>
<a NAME="ckd2cckd">
<a NAME="cckd2ckd">
<a NAME="fba2cfba">
<a NAME="cfba2fba">
<table>
<tr><td valign="top"><b>ckd2cckd  </b><br><b>cckd2ckd  </b><br>
<b>fba2cfba  </b><br><b>cfba2fba  </b></td>
<td valign="top">These utilities are deprecated. Use the <b>dasdcopy</b> utility instead</td>
</table>
<p>
<a NAME="cckdcdsk">
<table>
<tr><td valign="top"><b>cckdcdsk  </b></td>
<td valign="top"><em>[-v] [-f] [-ro] [-level] filename1 [filename2 ...]</em></td>
<tr><td valign="top">   </td>
<td valign="top">Check the integrity of one or more compressed files.
Recover damaged files.</td>
<tr><td valign="top">   </td>
<td valign="top">
<table>
<tr><td valign="top"><b>-v  </b></td>
<td valign="top">Display version and exit.</td>
<tr><td valign="top"><b>-f  </b></td>
<td valign="top">Perform check even if the <em>OPENED</em> bit is on.</td>
<tr><td valign="top"><b>-ro  </b></td>
<td valign="top">Open the file(s) <i>read-only</i>. The file will not be updated.</td>
<tr><td valign="top"><b>-level  </b></td>
<td valign="top">A number 1 .. 4 indicating the level of checking.<br>
<b>1</b> Minimal checking (default)<br>
<b>2</b> Medium checking. All track headers will be read.<br>
<b>3</b> Maximal checking. All track images will be read and uncompressed.<br>
<b>4</b> Recover everything</td>
</table>
</td>
</table>
<p>
<a NAME="cckdcomp">
<table>
<tr><td valign="top"><b>cckdcomp  </b></td>
<td valign="top"><em>[-v] [-f] [-level] filename1 [filename2 ...]</em></td>
<tr><td valign="top">   </td>
<td valign="top">Remove all free space from a compressed file or files.</td>
<tr><td valign="top">   </td>
<td valign="top">
<table>
<tr><td valign="top"><b>-v  </b></td>
<td valign="top">Display version and exit.</td>
<tr><td valign="top"><b>-f  </b></td>
<td valign="top">Perform compress even if the <em>OPENED</em> bit is on.</td>
<tr><td valign="top"><b>-level  </b></td>
<td valign="top">A number 1 .. 3 indicating the <a href="#cckdcdsk">chkdsk</a> level.</td>
</table>
</td>
</table>
<p>
<a NAME="cckdswap">
<table>
<tr><td valign="top"><b>cckdswap  </b></td>
<td valign="top"><em>[-v] [-f] [-level] filename1 [filename2 ...]</em></td>
<tr><td valign="top">   </td>
<td valign="top">Change the <i>endianess</i> or <i>byte-order</i> of a
compressed file or files</td>
<tr><td valign="top">   </td>
<td valign="top">
<table>
<tr><td valign="top"><b>-v  </b></td>
<td valign="top">Display version and exit.</td>
<tr><td valign="top"><b>-f  </b></td>
<td valign="top">Perform swap even if the <em>OPENED</em> bit is on.</td>
<tr><td valign="top"><b>-level  </b></td>
<td valign="top">A number 1 .. 3 indicating the <a href="#cckdcdsk">chkdsk</a> level.</td>
</table>
</td>
</table>
<hr noshade>
<p>
Greg Smith
<a href="mailto:[email protected]"><em>gsmith</em>@<em>nc.rr.com</em></a>
<p><center><hr width=15% noshade>
<a href="index.html"><img src="images/back.gif" border=0 alt="back"></a>
</center>
<p class="lastupd">Last updated $Date$ $Revision$</p>
</BODY>
</HTML>