Oldskooler Ramblings

the unlikely child born of the home computer wars

Vintage DOS Memory Conventions Explained

Posted by Trixter on August 20, 2018


Confusing expanded (EMS) memory with extended (XMS) memory happens all the time in the vintage PC/DOS computing hobby for those new to those terms.  Which is which?  What does my hardware support?  Do I need them?  This article will attempt to explain, as succinctly as possible, what each “type” of PC memory is and how it is used.  This article is not an extensive technical dive, but is meant to be an explanation of DOS memory conventions for the novice DOS user who is new to using DOS.

Protected Mode memory (DPMI, VCPI, DOS extenders, etc.) are not mentioned in this article because they typically don’t require any user intervention to work (ie. DOOM comes with DOS4GW.EXE and it just loads and works automatically).

Terminology used in this article

PC: Any IBM PC  or compatible system from the 1980s and 1990s running DOS.
8088: Shorthand for the entire 8086 CPU class, which includes the 8086, the 8088, the NEC V20, and the NEC V30.
640KB: The typical limit of memory DOS can access.  (Exceptions for DOS to access higher than 640KB exist, but are not mentioned to keep this article understandable for the layman.)

Quick history of memory on the PC

The 8088 CPU in the first IBM PC could access up to 1MB of memory.
Later, PCs built with the 80286 CPU could access up to 16MB of memory.
Later still, PCs built with the 80386 CPU could access up to 4GB of memory.

All PCs can directly access the first 1MB of memory.  Accessing memory beyond the first 1MB requires functionality only 80286 and later CPUs have.  Memory that extends past the first 1MB is called Extended Memory.

Because 8088 CPUs can’t access Extended Memory, special memory boards were created for them that fit into their system expansion slots.  These boards contain extra memory visible somewhere in the first 1MB, where the 8088 can see it.  The memory provided by these special expansion boards is called Expanded Memory.

Memory types in detail

EMS, XMS, conventional memory, UMBs… what does it all mean?  Here are some definitions:

Conventional Memory

What is it?  The first 1MB of memory visible to all 8088 and higher CPUs.  DOS is loaded into the area from 0-640KB and manages it.  The area above that, from 640KB to 1024KB (1MB), is typically used by the system hardware and not generally available to DOS programs.
How do DOS programs use it?  DOS provides function calls for allocating, resizing, and deallocating memory blocks for program use.  (Alternately, programs can directly access any portion of the first 1MB of memory they want to, although doing so can be risky if the programmer is inexperienced.)

Extended Memory

What is it?  Memory visible beyond the first 1MB, usually physically located directly on the motherboard, but sometimes provided by adding memory cards.
How do DOS programs use it?  Via an API called the eXtended Memory Specification, also called XMS.
What provides XMS?  XMS is provided by a driver called HIMEM.SYS, loaded by CONFIG.SYS at boot time.
How does XMS work?  XMS provides function calls that DOS programs can use to copy data between conventional memory locations and extended memory locations.

Expanded Memory

What is it?  Memory provided by special expansion cards.
How do DOS programs use it?  Via an API called the Expanded Memory Specification, also called EMS.
How is EMS provided?  Via a driver provided by the expansion card manufacturer.  Each card has a different driver and you must use the correct one for your card.  The driver configures the card for use, and provides EMS function calls that DOS or programs can use to access the memory on the card.
How does EMS work?  EMS boards provide a small “window” into the memory they contain, and this window is located somewhere in the first 1MB of RAM where 8088 CPUs can access it.  To access more memory, programs issue EMS function calls that “move” the window to a different area on the card, changing what portion of the larger memory shows up in the small window.

A quick note about EMS:  If you want to run a program that uses EMS, but you don’t have an EMS board in your computer, don’t fret: You can emulate EMS on any 80386 or higher.  This is done by loading a memory manager such as EMM386 or QEMM.  One of the services provided by a memory manager is to section off a portion of Extended Memory and present it in response to EMS function calls, like a real EMS board would.

Upper Memory

What is it?  The upper portion of Conventional Memory located between 640KB and 1024KB (1MB).  There is typically no user-accessible memory in this area, but 80286 and higher systems can relocate portions of Extended Memory into this area for use by DOS programs.
What is it used for?  The total amount of upper memory available is typically very small, between 64KB to 128KB, but this amount can still be useful for loading small memory-resident programs or drivers outside of Conventional Memory.
How do DOS programs use it?
  XMS provides functions for allocating and deallocating memory in this area in units of “blocks”, called Upper Memory Blocks, or UMBs.
How are UMBs provided?  For 80386s and higher, DOS comes with HIMEM.SYS and EMM386 which, loaded together, provide UMB functionality.  For 80286s and lower, there are programs (such as QRAM or USEUMBS) that use “shadow ram” functionality provided by the 80286 system’s chipset to perform the mapping.
How does upper memory access work?  Once a program is provided an upper memory block via a UMB functional call, it can be accessed the same way conventional memory is accessed, ie. directly.

Guidelines to running DOS programs

  1. First off, simply try to run a DOS program.  Most run without needing any special memory configuration.
  2. If you have an 80286 or higher, always load HIMEM.SYS in your CONFIG.SYS file.  There’s no harm in installing it even if programs don’t use it.
  3. If you want to load a single program on a 386 or higher to manage all of this for you, install QEMM.  QEMM will manage all of your memory and provide your programs with either XMS or emulated EMS based on what each program asks for.
  4. Finally, if a program claims to support both EMS and XMS, choose EMS.  EMS is faster than XMS.

3 Responses to “Vintage DOS Memory Conventions Explained”

  1. Trixter said

    For those who would be quick to point out things I “missed” in the article, I left them out on purpose so as to make it as clear as possible for newcomers. But if you want them mentioned anyway:

    – UMBs can be provided on 8088 systems too if there is physical memory located there. Some Juko motherboards, and homebrew memory cards like the lo-tech series, can do this. USEUMBS can then map these.
    – While a 386 can address 4GB of RAM, there was no 386 system in the world with that much physical RAM on it. This was due to hardware limitations and cost reasons.
    – EMS is faster than XMS because XMS is a copying API — you copy stuff to/from extended memory. EMS remaps memory, so a single call makes the memory just “show up” in the page frame (“window”).
    – There is a set of INT 15h BIOS calls that can manage extended memory, but these are not consistently supported and hardly anything used them. XMS is the standard.

  2. > “4. Finally, if a program claims to support both EMS and XMS, choose EMS. EMS is faster than XMS.”

    …if the EMS is old school, i.e. memory provided by an actual EMS expansion card.

    However, if you’re using XMS to *emulate* EMS (by running EMM386.EXE or a similar program), then that emulated EMS is slower than using XMS directly. (Correct me if I’m wrong on this, because I don’t have benchmarks now, but I’m pretty sure I’m right.)

    (PS for the record: Like the EMM386 name suggest, XMS-based EMS emulation was only a thing on 386+ hardware though. It wasn’t an option on pre-386 PCs. If you wanted EMS on 8088/80286 PCs, you went and bought an actual EMS hardware expansion card.)

    • Trixter said

      EMS is always faster than XMS in these cases:
      – EMS is provided by a hardware board, for reasons explained above
      – EMS is provided by a memory manager running on a 386+, because the memory manager can page memory in and out of the EMS page frame using 386+ global descriptors
      – The calling program is a 16-bit real-mode program

      EMS is slower than XMS in these cases:
      – EMS is being provided by an “emulation” driver that relies on some other storage layer, such as XMS or a large paging file on a hard disk
      – The calling program is a 32-bit protected-mode program (which bypasses EMS and XMS completely)
      – The calling program’s usage pattern involves repeatedly *writing* to non-conventional-memory blocks larger than 64K. The only use case I can think of for this would be a disk cache, virtual memory system paging activity, or print spooler, but this is really a wash in practice.

      XMS v3.0 has functions to lock and unlock memory blocks, and return locked blocks’ addresses as a 32-bit pointer, but there were very few use cases where this made any sense — most DOS programs would use a protected-mode DOS extender and just use the flat address space natively.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: