ChaOS Home ChaOS Source Notes ChaOS Source Index ChaOS Downloads CTPP Home
Current Diary Diary to Jun 2010 Diary to Mar 2010 Diary to Dec 2009 Diary to Aug 2009 Diary to Apr 2009
ChaOS Diary - monoblog and links to reference documents
Many golden nuggets lie herein.
30/11/2008 HTTP/TCP Added tx queuing to TCP, which holds data segments until connection is established. So after the empty ACK (23/11/08) a poll to the tx queue causes the opening command segment to be sent. At least now I have a working testbed which shows how servers react to various situations. Basic TCP active open( ) and send( ) working adequately. Completed router HTTP login with username and password encoded in base64. Connections closed by FIN time out after a couple of minutes, after which time it seems OK to reuse a port number. Now reducing the receiver window size to achieve inbound flow control, needed to implement a receive( ) function with a finite buffer size. Whilst Zen Apache server correctly observes window size (and stops sending data when window size is zero) , my router HTTP server keeps sending data regardless of window size. Which is a pain.Ask for 200 bytes, get 458. Either I always have to allocate extra space in the user buffer on a receive( ) command (just in case the target server is a rogue), or I need to make the TCP queue the extra bytes and give them up on a subsequent call to receive.
28/11/2009 NJOB Not really anything to do with ChaOS, but Gordon Brown and Alastair Darling just decided to give businesses just 10 days notice of a change in the standard rate of VAT from 17.5% to 15%. Fortunately I only needed to add a new variable to NJOB to allow the VAT rate to be changed, and 4 lines of code. But it is costing other small businesses thousands of pounds to change their software.
28/11/2008 HTTP/TCP New TCP session model created in IP.DRV, based on EFSM. The Zaghal and Khan EFSM does not model the PSH control bit, nor does it model user data being sent to the server immediately after receiving SYN|ACK. So I am wading through RFC793 again. I have just realised there is a simple HTTP server in the router, so I will use this as a testbed to avoid sending garbage to my ISP.
27/11/2008 Dynamic-linking There is a latent bug in the type matching system; it happens when dynamic-linking functions containing a pointer-to-structure as an argument, when that structure type contains a function pointer type which also uses a pointer to the same structure type as one of its arguments. It results in infinite recursion in findequivalenttype( ). I work around the problem at the moment by typecasting, i.e.cheating the type system, but it needs to be fixed because it does not cause a debug exception when it happens.
25/11/2008 TCP I only discovered RFCs a couple of months ago. They are the foundation stones of the internet, but a lot of them are irrelevant to the internet of today. You may not need to be a rocket scientist to understand them, but for some, like RFC793 (September 1981) it helps. I'm now using Zaghal and Khan's EFSM/SDL Model (July 2005), which is a clearer description of TCP protocol and its links to the user.
Had a play with the November 2008 ISO download, and added some release notes to the download page.
23/11/2008 (late) HTTP/TCP TCP Session control is largely buried in the mists of time, but a TCP session starts with a client sending a SYN packet to server, which responds with a SYN|ACK, then an ACK from the client completes the negotiations. Zen server opened a TCP session with ChaOS when I included data (HTTP GET command) on the ACK packet. This kind of stuff is not in any books. All day I had been sending an empty ACK, with various permutations in the packet header. TCP source port numbers from aborted sessions earlier in the day appear to have timed out and are usable again.
This really is a large step forward for ChaOS. Quite scary seeing the CTPP homepage bubbling into the ChaOS network buffers, even though I have no code written to assemble the data on the incoming packets
23/11/2008 HTTP Not quite so easy as DNS, which can be done with UDP packets, HTTP requires a TCP session and the Zen server blocks my source port each time I try to connect with experimental code. Back to the drawing board.
22/11/2008 DNS Revisiting ChaOS network protocol projects. Connection to the internet through a router is simpler than I had imagined, managed to construct a successful DNS query packet at the fourth attempt. DNS response decoding now complete, looking forward to establishing a HTTP session running in ChaOS with the CTPP website.
20/11/2008 November 2008 ISO download Uploaded nov2008test.iso and nov2008test.zip, a preview of the multiboot ISO to come which will be much larger.
18/11/2008 PS2 Cured an old problem which locks PS2 keyboard on some mainboards (data waiting on port 0x60 with no interrupt to clear the controller) caused when trying to initialise non-existent PS2 device (keyboard controller aux device i.e. PS2 mouse). Fixed by aborting ps2init( ) when (*(UC*)(0x410))&4 (BIOS equipment byte- PS2 device present) is zero. The problem can also be solved by reading the spurious byte from port 0x60. I know the problem might be better addressed by keeping interrupts on during the whole of ps2init( ), but this would mean creating a buffer for the PS2 aux device interrupts before the device has been identified. On my 845 mainboard BIOSequipmentbyte&4 is set whether or not a mouse is plugged into the mainboard on powerup, but initPS2( ) seems to avoid a keyboard lockup.
CC ChaOS compiler A nasty bug has hampered efforts to move OS startup code to a separate file. I first encountered this problem when writing 2005 demo. The ChaOS inline assembler uses the 'C' compiler getop( ) function, which is how the assembler hooks into the 'C' symbol table, and can use integers generated by #defines in the 'C' source. But to allow the assembler to encode forward code jumps, a non-existent symbol cannot be an error (it may be a code label which appears later in the compilation). In the case of this bug, the assembler uses an integer macro which is never defined, and compiles as as zero integral value with no error. But this is the macro used to set flags in the processor CR0 register! (a zero in CR0 switches the processor to real mode). Ouch! I had merely omitted the header file containing the processor flag macros from the startup file. Added a couple of lines of code to the compiler to stop this happening again!
17/11/2008 Bootstrap Early versions of ChaOS used a processor reset to re-enter real mode and return to DOS - this was the only way to get the old 80286 processor back into real mode, and is a hangover from the Lotti Design System. To return to a useful address, 80286 BIOSes provided a return-from-reset mechanism which I call shutdown(10), which can be used to return to the bootstrap. From 80386 onwards, real mode can be re-entered by clearing bit 0 of register CR0, and this is how ChaOS real-mode callbacks work. I've been aware for a while that shutdowns don't work on the Stylistic ST4110 (an infuriating piece of kit!). The ChaOS bootstrap can be re-entered by a call to realmodefarjump( ); the BIOS bootstrap can be re-entered using a call to realmodeinterrupt( ) (interrupt 0x19 or 0x18). So shutdown(10) (which I have used since 1988) will now become obsolete.
16/11/2008 Bootstrap Testing the November 2008 demo kernel on the Stylistic ST4110 encountered a weird problem with CGA text mode 3, requiring another call to Int 0x10/3 and a time delay before CGA memory at 0xb8000 was available to read cycles. The Stylistic seems unable to handle a call to video BIOS immediately before the switch to protected mode. Bootstrap now modified to pass the active graphics mode in BOOTINFO, and it is now up to the operating system to decide whether a further video mode change is required. A real-mode callback to the video BIOS at the start of osmain( ) solves the problem on the Stylistic, as well as paving the way for some ChaOS demos which run entirely in graphics mode.
15/11/2008 November 2008 ISO download ChaOS V1.01.24366 is 79 source files, demo kernel now down to 467k data and code from 46 source files (not including header files), 31 of which are shared with current ChaOS version, and much more stable than the october 2008 iso download which is based on 2005 ChaOS files.
12/11/2008 November 2008 ISO download Trying to find some interesting bootable images to put on the next ISO download. ChaOS V1.00.06902, (24.2.2002) boots up fine, even though it was developed on an old 350MHZ Pentium, I will include that, it will need to occupy its own partition on the ISO CD because the bootstrap (LOADER.BIN) has changed much since then. I also need some compact demo images, (otherwise the download will be too big), so I am hacking down ChaOS 2008 to produce a small working kernel on which to base a series of demo images, such as a memory browser, PCI and PnP device browser, and a BIOS interrupt despatcher to allow the user to play with BIOS interrupts using the ChaOS pmrm switch.
8/11/2008 Fujitsu Stylistic ST4110 Cardbus interrupts Finally solved the puzzle of the Stylistic ST4110 Cardbus slot - on boot up, the PCI IRQ routing register is wrong, and the device memwindow address at PCI:0x10 is blank (though PCI:0x44 is set to 0x3e1 to enable the legacy ExCA register set). I had tried setting the memwindow address to a location just above TOM (top of memory), but code which happily runs Cardbus on my other computers would not work on the Stylistic. However when the memwindow is set to an address inside the PCI Host non-prefetchable memory window, everything suddenly works. Though it is obvious that memory-mapped registers need to be in non-prefetchable memory, this is the first PCI device I have encountered where the BIOS does not set the register window. Stylistics are just so quirky.
1/11/2008 Cardbus IRQ routing PC card interrupts now despatched from inside Cardbus interrupt handler.
31/10/2008 Cardbus/PCMCIA IRQ routing Returning to the problem plaguing my Fujitsu Stylistic ST4110 project - apart from the PCI INT# routing registers being wrong on the Fujitsu, regarding interrupts from devices in Cardbus/PCMCIA slots using the TI1225 - despite what the datasheets say, and hours of experiment I can find no way of routing Card Status Change (CSC) and PC Card interrupts to separate IRQs. ChaOS handles shared IRQs, but identifies the hardware needing to be serviced by sending DM_TRIGGER messages to the hardware drivers. Unfortunately there is no flag bit on basic IDE devices to indicate that an interrupt is pending, so the inbuilt ChaOS IDE/ATA driver has no way of reporting the interrupt trigger to the IRQ despatcher. Interrupts from devices in Cardbus slots are flagged at PCIcfg:0x91&1, so there is a way to fix this. Equally, IDE controllers with Bus Master registers do have an interrupt flag bit. Given that these interrupts are on the same IRQ, it may be easier to despatch the PC Card interrupt handler from within the Cardbus interrupt handler.
26/10/2008 ISO CD download New ChaOS download page created with link to oct2008.iso, which is the 2005 floppy disk download converted to bootable (EL TORITO SPECIFICATION) ISO CD format. Some CD creation programs will not recognise the ChaOS multiboot CD ISO format, so I have added a copy of CDburnerXP to the downloads page, a great piece of software, it is free, and handles the ChaOS ISO format.
22/10/2008 HTML Source Files Although HTML tags pass through the compiler without generating code, they do slow the compilation pass, so I have made a further change to syntax, i.e. where the first character of a source line is '<', the whole source line is discarded by the compiler. The HTML prologue and epilogue lines at the start and end of the source files no longer affect compilation speed.
21/10/2008 HTML Source Files Files for ED - source file editor program uploaded, plus a couple of header files
16/10/2008 HTML Source Files New #include directive syntax: Compiler modified to recognise #include{....} as well as #include <...> and #include "...". Using braces { } instead of < > to delineate include file names means HTML browsers will not try to interpret <filename> as a HTML tag.
It was surprisingly easy to implement the HTML source file enhancements.
15/10/2008 HTML Source Files Compiler now modified to use .HTM source files, uploads to the source file index will now be in the new format. Hyperlinks are best located inside comments where they are not scanned by the compiler, so there is little need for the code which allows hyperlinks inside expressions - but I have left it there for the moment, it may be useful one day. The simple HTML wrapper placed around the ChaOS source now includes a <chaos...> tag, where I can put attributes if necessary.
On reviewing these files when uploaded to the website, I have just realised that #include <.....> directives in the source files do not pass through HTML browsers, and appear blank (you idiot!). I will have a think about this - a small modification to the #include syntax is needed for the HTML source file idea to be complete. (NB fixed by new #include {...} syntax 16.10.2008).
10/10/2008 PCI and HTML Source files Following on from diary 18/9/2008, Generic hst (PCI bridge) now running at DEV[0]. hst sets a global variable basicpci if a PCI bridge is detected. Now all devices can be PCI-aware during startup.
Had a play with the compiler, to see if HTML tags can be embedded in the source code. Surprisingly, simple HTML tags such as <BODY> </TITLE>, when outside data definitions and functions, pass through the compiler without generating any code, (now that's lucky!). I haven't bothered to check why - because the HTML wrapper for the ChaOS source files to display nicely over the internet is so simple:
<HTML><HEAD><TITLE></TITLE></HEAD><BODY bgcolor="#99ccff"><PRE>
........source file
</PRE></BODY></HTML>
Hyperlinks are a bit trickier - I have added a bit of code to the expression analyser which checks for an HTML <A> tag, which then scans for the </A> before handing control back to the parser. Because <A> tags can point to other documents, I can now put links in the source code to the technical references used to build the code.
This idea needs a little more work, I need to be able to place hyperlinks inside comments too.
18/9/2008 PCI I had started to develop some DEVs (loadable device drivers) specific to the various mainboards running ChaOS - but for the PCI IRQ routing job, it seems the registers needed are found in the same place on mainboards from different vendors. So I will adding a generic PCI device at the head of the ChaOS device list, to run before anything else. Detecting a PCI bridge is pretty simple:
outpd(0xcf8,0x80000008); //PCI Dev 0, Function 0 register 2 Class and Sub-class codeunsigned int n=inpd(0xcfc);if((n&0xffff0000)==0x06000000){//we have a PCI host bridge}
The PCI IRQ routing registers for INT#A-INT#D are found at Device 31:register 0x18, so
outpd(0xcf8,0x8000f860);n=inpd(0xcfc); //to readoutpd(0xcfc,n|0x80000000); //to write
18/9/2008 Problem encountered on Fujitsu Stylistic ST4110 with regard to incorrect PCI IRQ routing for the Cardbus slot by BIOS on boot up has caused me to go off on a bit if a tangent. I had previously used the IRQ reported in the PCI:0x3c byte as gospel when assigning IRQs to PCI - but this value is just a read- write byte that BIOS (or anyone else) can use to store a number - an unrouted IRQ stored here causes ChaOS to install an interrupt handler on the wrong line. Real PCI IRQ routing is through the PCI:0x3d byte (INT#A -INT#H) and the PCI IRQ routing registers on the PCI-ISA bridge, usually found at PCI:0x60 and PCI:0x68. These routing registers can also direct the PCI IRQs to APIC (Advanced Programmable Interrupt Controller), if present. I've got a CF-GPS running in the Stylistic Cardbus slot, it's basically RS232 but the receiver FIFO keeps overflowing. I think this is because I am using the same IRQ to handle both Cardbus events (card insertion, ejection etc) and interrupts from the device in the PCMCIA/Cardbus slot. The device inserted may require a different IRQ trigger type on the line - Cardbus/PCI needs level-triggering whilst the device in the slot may need edge-triggering. This in set in the ELTR registers at 0x4d0-0x4d1. I am currently reworking the ChaOS IRQ resource handling, to check and modify PCI IRQ routing as necessary. Had a good play with the APIC too - you can run some interrupts through the 8259's whilst running others through the APIC. Had ChaOS running all IRQ's on the APIC, no problem - the 8254 timer (IRQ0 since the year dot) is connected to IRQ2 on the APIC - that foxed me for a while. As the Fujitsu Stylistic ST4110 has no APIC, the job in hand is the PCI IRQ routing.
2/9/2008 Had a look at the problem with the 2005 Windows-hosted ChaOS download under Windows XP - Microsoft in their wisdom prevent absolute floppy disk sector writes in Windows XP. But the old floppy downloads OK on Windows98 or Linux. see downloads page. Any problems send an email. Update October 2008: new ISO CD downloads page added
2/9/2008 Created this page