Problem Set 3

		 Harvard Extension School CSCI E-92:
		   Principles of Operating Systems

			     Spring 2024

	  Due: March 17, 2024 at Midnight ET (Eastern Time)


Total of 330 Points

1. (330 Points) Input/Output Programming.  Design and code a
device-independent system that allows input and/or output operations
to a variety of hardware devices.  For this problem set, the supported
devices will be the LEDs, the pushbuttons, and a FAT32 file system
running on the microSDHC system.

Porting myMalloc and myFree to the K70

This problem set should be implemented on the Freescale K70 Tower
system.  Therefore, your current myMalloc and myFree functions need to
be ported from our cscie92.dce.harvard.edu Linux EC2 instance to the
K70 environment.  Keep in mind that the K70 has 128K bytes of static
RAM and, therefore, the built-in malloc call supports a maximum size
that is substantially less than 128K bytes.  It is likely that the
maximum size memory request for a successful system malloc call will
be about 64K bytes.  Depending on how much RAM the system and your
program uses, you may find that you are able to allocate substantially
less than 64K bytes of memory.

Using the UART serial port instead of the console

You are required to use UART serial port I/O for your shell's
interaction with the user.  That is, all of a user's usual interaction
with your operating system will take place over the UART-driven serial
port rather than through console I/O (semihosting).  Feel free to use
the uart.h and uart.c code from the class web site to accomplish this.
You should configure the UART for 115,200 baud with eight data bits,
one stop bit, and no parity.  Your terminal emulator (such as
SecureCRT, Minicom, PuTTY, Tera Term, Serial, HyperTerminal, RealTerm,
etc.) should be configured to send just a carriage-return when the
"enter" key is pressed.  And, your terminal emulator should be
configured to: (1) return to the beginning of the current line when it
is sent a carriage-return (\r, ^M, or control-M) and (2) go to the
next line in the same column when it is sent a line-feed (\n, ^J, or
control-J).  In addition, your terminal emulator should be configured
to *not* perform local echo; this means that characters typed will
*not* automatically be displayed on the terminal emulator's screen.
Your fgetc routine for the serial port will have to output characters
using fputc if it wants to have those character displayed.  These are
probably the default settings for your terminal emulator.  We will be
using this configuration for all serial port interactions in future
assignments.

If you wish, in addition to the UART serial port I/O, you can use
console I/O (semihosting) for outputting debugging messages.  Any
output over console I/O (semihosting) must be able to be disabled and
removed from your program using an #if, #ifdef, or #ifndef
preprocessor directive.  If you are using KDS as your development
environment, keep in mind that console input under KDS is not
implemented.

Any use of printf, fprintf, and similar functions in your OS (except
for debugging messages) will need to be changed to snprintf or
vsnprintf calls -- we will refer to these functions as "formatted I/O
to string" functions -- to produce formatted output into a string that
can then be output using UART I/O.  In addition to snprintf and
vsnprintf, you may use the sscanf and vsscanf calls as additional
formatted I/O to string functions.

If you are *not* using the formatted I/O to string functions and you
are not using console I/O, your projects should now be built using "No
I/O" under "I/O Support" in the "Language and Build Tools Options"
screen in CodeWarrior or "-specs=nosys.specs -specs=nano.specs" in
"Other linker flags" in KDS.  If you are using the formatted I/O to
string functions, your projects should still be built using "Debugger
Console" under I/O Support in CodeWarrior -- this includes the full
Embedded Warrior Library (EWL) which has support for snprintf, etc.
And, if you are using console I/O output for debugging or for
catastrophic messages, then you should enable the "Debugger Console"
in CodeWarrior or semihosting in KDS ("-specs=rdimon.specs
-specs=nano.specs" in "Other linker flags" in KDS).  Also, for KDS
"semihosting" needs to be enabled for the debugging interface you are
using -- either P&E or Segger.

Device-independent I/O

You will be implementing a layer in your I/O system that allows users
to write programs that can function independently of which specific
device is being used.  This device-independent I/O is used, for
example, in a shell, when we can invoke a "cat <args>" command to
output to stdout the contents of each of the files/devices listed as
<args> on the command line in order sequentially.  That "cat" command
doesn't need to know about the specifics of the files/devices; rather,
it performs all of its input/output operations using
"device-independent I/O."

Your device-independent I/O system should be based on byte-size input
and output operations, customarily named fgetc and fputc,
respectively.  These calls require a file to be opened before
operations are performed and closed after operations are performed.
The functions to perform those operations are customarily named fopen
and fclose, respectively.  Traditionally, fopen returns a pointer to
an object that can be used to control the input and output operations
-- we'll refer to this pointer as a stream.  Streams in the ISO C
Standard are declared as "FILE *" -- that is, as pointer to FILE.  In
the Standard, there are some predefined streams (stdin, stdout, and
stderr) that have predefined values and can be used without requiring
calls to fopen and fclose.  If you choose to implement these, they
should be opened by your operating system before starting to run the
"application."  Of course, your design does not need to use this
specific mechanism or these same names nor does your design need to
use exactly the same parameters or return values.  However, unless you
have another design that you'd like to pursue, we encourage you to
implement the usual ISO C/POSIX standard calls.  As defined in ISO C,
the values of the three standard streams are as follows: stdin has the
value 0, stdout has the value 1, and stderr has the value 2.  One
implementation approach is to create and utilize each streams'
pointer-to-FILE ("FILE *") as an index into an array of open streams.
That array would be located in each process' PCB.  Thereby, as
appropriate, each process would have its own set of open streams.
Each entry in that array could be a pointer to a "struct stream."

You need to allow any 8-bit character in your fgetc and fputc calls.
This is especially true for the FAT32 file system and some other
devices we will add later.  Therefore, in addition to printable
characters, you need to allow any byte value with values from 0 to
255, inclusive, to be valid characters.  If you want to have a return
value from fgetc that indicates end-of-file (EOF) or some other
exceptional condition, you will probably want fgetc to return an int
and use negative values for those exceptions.

You are welcome to have a fixed maximum number of concurrently open
streams for each process.  You must allow at least 32 streams to be
open at the same time for each process.  (Therefore, in addition to
stdin, stdout, and stderr, you must allow at least 29 additional
streams.)

The stream would contain all information necessary to perform the
fgetc, fputc, and fclose operations correctly.  For the LEDs or the
pushbuttons, the struct stream might include a pointer to a structure
that has information about that kind of device (let's refer to this as
the "struct device") and also a second field that indicates which
specific device is open.  So, there might be three "struct devices"
created: one for the LEDs, one for the pushbuttons, and one for the
FAT32 file system.  For the LEDs, the second field might indicate
which LED is open (the orange, the yellow, the green, or the blue).
For the pushbuttons, the second field might indicate which pushbutton
is open (SW1 or SW2).  The "struct device" might contain function
pointers to routines for that specific device to perform fgetc, fputc,
and fclose for that kind of device.  Those routines might be
parameterized by the second field in the "struct stream" to identify
the specific device (e.g., the specific LED or pushbutton).  For the
FAT32 file system, more information is needed to remember the state of
the operation.  You can think of the pointer to "struct device" as the
UNIX major device number and the second field as the UNIX minor device
number.

As mentioned above, all associations for streams that have been opened
should be stored in the PCB struct that was created for Problem Set 2.
When processes are fully-implemented in Problem Set 6, these stream
associations will be per-process (rather than global).  This will
allow, for example, each process to have its own stdin, stdout, and
stderr.

You should design your own methodology for naming devices and files.
For example, if you want to model your system after a Windows-like
naming convention, you might name devices with a letter (or a short
string) to specify the device, followed by a colon.  Similarly, a
device that supports named files might use a name which starts with a
letter (or a short string) to identify the device, followed by a
colon, followed by the file name.  The FAT32 specification determines
which characters are allowed in file names; you must enforce those
constraints.  As an alternate to Windows-like names, you might decide
to model your system after a Unix-like naming convention.  Devices
like an LED might be named /dev/led/green, /dev/led/blue, etc.  Such
device names might or might not actually exist as files on the
microSDHC FAT32 file structure.

The FAT32 file system

For details on the FAT32 file system and the lowest-level functions
that you need to implement, please refer to the FAT32 File Structure
slides.  For the FAT32 file system, those lowest-level functions will
be used as the basis upon which you will build your device-independent
byte-oriented I/O system.

In order to have an efficient implementation for reading from and
writing to files on the FAT32 file system, we are requiring you to
implement a cache for at least a single sector that would contain the
last referenced sector from the file system.  That single sector cache
would be referenced by functions to read or write a sector in a file.
Associated with the cache would be variables that indicate: (1) if the
cache contains a "valid" entry (initially the cache would not be valid
because no sector would initially be in the cache), (2) the "sector
number" of the sector in the cache, (3) if the sector in the cache was
"modified" after being read from the file system.  An additional
function would be added to flush the cache -- flushing is an operating
that writes the cache contents to the file system if it has been
modified.

In addition to supporting the fopen, fgetc, fputc, and fclose
device-independent functions on top of your FAT32 file system, you
also need to support device-independent create and delete functions.
The create function will create a named file on the FAT32 file system
and the delete function will delete a named file from the FAT32 file
system.  These functions will be based on the functions mandated in
the FAT32 File Structure slides.  You are welcome to implement other
device-independent functions for additional capabilities of your
system.  These might include dealing with the current working
directory (chdir), creating and deleting directories (mkdir, rmdir),
etc.

Your design should specify the behavior when certain operations are
attempted.  These include: (1) Can any device be opened simultaneously
more than once?, (2) Can any file be opened simultaneously more than
once?, (3) What happens if a file already contains some bytes and an
fputc is issued to that file?  If in the beginning or middle, is the
file truncated at that point or does the fputc simply overwrite bytes?
If at the end, is the file extended?

Your implementation need not follow the same specifications as the
Unix system calls of the same names, but it may be instructive to look
at the documentation for those calls.  Feel free to simplify, alter,
and/or extend the Unix calls as appropriate.

Shell extensions

So that we are able to test your device-independent I/O system, you
must port your shell implementation from Unix to the K70 and add
commands to your shell that call the mount, umount, setcwdtoroot, ls,
fopen, fclose, fgetc, fputc, create, and delete system calls.  Also,
if appropriate, add a command for each optional system call that you
implement.  Each command should issue the corresponding system call
once.  Don't try to get your date command working at this point in
time.

Because the fgetc system call may return a non-printing character, the
fgetc shell command should translate all non-printing characters to a
printing form before outputting the byte read by the fgetc shell
command.  The printing characters have ASCII values from 32 decimal
(0x20) to 126 decimal (0x7e), inclusive.  Characters with ASCII values
from 0 to 31 decimal (0x1f), inclusive, and from 127 decimal (0x7f) to
255 decimal (0xff), inclusive, should be printed in hexadecimal with a
prefix of "0x" (without the quotes).

We also want you to implement a "cat" shell command that takes a FAT32
filename as its only argument.  The "cat" command will open the file
using fopen, read each byte of the file one at a time using fgetc and
output each byte after being read to the UART.  The "cat" command will
end when the end of the FAT32 file is reached.  At that point, the
command will call fclose to close the file.

As always, all code should be heavily documented and commented.  All
return codes and command line arguments should be checked and any
errors appropriately handled.  The clarity and design of your code
will count!

(10 Points) Extra credit to be used for the programming portion only:
Allow nested directories.  Must implement dir_set_cwd_to_filename,
dir_create_dir, and dir_delete_dir.

(20 Points) Extra credit to be used for the programming portion only:
Allow long filenames.

(Varying Points) Extra credit to be used for the programming portion
only: Implementing creation date and time.  Implementing last access
date.  Implementing "full" directory listing in dir_ls.  Implementing
both dir_ls_init and dir_ls_next.  Implement interfaces to set and
retrieve file attribute DIR_ENTRY_ATTR_READ_ONLY and to respect
attribute DIR_ENTRY_ATTR_READ_ONLY in file_putbuf.


			Last revised 6-Feb-24