COLLECTED BY
Organization:
Alexa Crawls
Starting in 1996,
Alexa Internet has been donating their crawl data to the Internet Archive. Flowing in every day, these data are added to the
Wayback Machine after an embargo period.
this data is currently not publicly accessible.
The Wayback Machine - https://web.archive.org/web/20080228231933/http://www.trnicely.net:80/misc/vista.html
Windows Vista restricts non-Win32 apps to 32 MB of memory
Thomas R. Nicely
Copyright � 2008 Thomas R. Nicely. All rights reserved.
This document may be reproduced and distributed for
educational and non-profit purposes. No warranties expressed or
implied.
Last updated 0030 EST 26 February 2008.
Original posting 0100 EST 8 March 2007.
Executable images created for the DOS/Wintel environment, using the
GNU GCC compilers and language standards (but not linking to the
Win32 API), are subject to failure (or performance degradation) when
executed in Microsoft Windows Vista, because Vista arbitrarily
restricts the memory space for the GCC executable to 32 MB
(33,554,432 bytes). Attempts to allocate more memory than this using
the malloc(...) function (or related functions, such as calloc(...))
will fail. This limitation applies whether the application is
executed with the Run command, within a Command Prompt
box (DOS box), or with the Start command. This limitation does not
appear in Windows XP, Windows 98SE, or standalone DOS; the exact same
executable, running under Windows XP SP2 or Win98SE, is capable of
allocating several hundred megabytes of physical memory (if present
on the machine). The limitation appears to apply to any compiler,
linker, or executable not conforming to Microsoft's proprietary
Win32 API.
The operating environment exhibiting this incompatibility in Vista
includes GNU GCC C 4.12,
DJ Delorie's DJGPP 2.04 beta (11/30/2003),
and Vista Home Basic 6.0.6000 running on a Dell E520 with a
Pentium D 915 2.8GHz dual core processor, 1 GB of 533 MHz
DDR memory, 800 MHz front side bus, and SATA drives.
The limitation appears with or without the use of executable
compression, and with or without the use of the DJGPP DPMI
servers CWSDPMI (Charles W. Sandmann), CWSDPR0, CWSDSTUB,
and PMODE/DJ (which are apparently ignored in any case, in favor
of the native Windows DPMI server). The system is operated as a
standalone, single-user machine, logged on as Administrator (Command
Prompt run as Administrator), with no Internet connection. Nearly all
security features are disabled, including User Account Controls,
Windows Firewall, Windows Defender, and (except for a warning message)
Security Center.
Essentially the same limitation applies to source compiled with the g77
Fortran compiler; one difference is that the code will usually fail
immediately with the error message "Load error: no DPMI memory"
if total memory requests exceed 32MB. I do not know if the problem
appears in other environments (e.g., other editions of Vista or codes
cross-compiled for DOS/Wintel with other versions of GCC). Proprietary
compilers which inherently convert code to the Win32 API (such as Visual
C++ or even Borland C++ 4.52 with PowerPak) do not exhibit this limitation;
they convert calls to the standard "C" malloc(...) to calls to
Win32 API functions such as GlobalAllocate(...) or VirtualAllocate(...),
which are then linked in at run time from Microsoft system DLLs.
I suspect the real problem may be that Vista is treating any
application that does not call the proprietary Win32 API (either directly,
or through compiler or linker translation) as a "16-bit"
application, and is then applying the 32MB limitation for
"security" reasons. The presence of what appears to a 16-bit
stub loader at the beginning of the GCC executables may also be an issue
for Vista, although it is no problem for XP or Win98SE.
Since the problem does not appear when the same executable image is
run under Windows XP or Win98SE, the problem clearly lies with Vista.
A simple test code is provided in the file
vista1.zip (39K), which includes the
source code vista1.c and a DOS/Wintel executable vista1.exe
compiled with GCC C 4.12 and DJGPP 4.12 under DJGPP 2.04 (no
compression or DPMI stub). The test code attempts to allocate
a large chunk of memory (40,000,000 bytes), using malloc(...);
if it fails, it repeats the attempt, decreasing the amount by
1,000,000 bytes each time. The code also writes to the memory,
then reads it back to verify the integrity (one compiler
generated code in which malloc reported success, but the memory
could not in fact be accessed). Results are reported to the console.
Run it from a Vista Command Prompt (DOS box), with the Run
command, or with the "Start" command; the results will
be the same---failure until the request is decreased to
33,000,000 bytes (or possibly less). However, if the same code
is run in a Command Prompt session in Windows XP or Win98SE, or
in a standalone DOS session under Win98SE, the allocation will
be limited only by the available physical memory. The code can
also be run with a command-line parameter, e.g.,
"vista1 100", which will begin the allocation attempts
at 100 million bytes.
For many applications, the problem may not be significant. However,
much of my work requires access to very large amounts of memory
(arrays of several hundred megabytes in some cases), and I feel sure
this is true for other developers as well. Even if all your work
is normally done on non-Windows platforms, it would still be unfortunate
to lose as a distribution base all of the Windows-based systems on
the planet---currently about 90 % of all systems, and more than likely
nearly all of that base will eventually be converted to Vista.
I spent several weeks looking for a fix for the 32MB memory
limitation, without success. I have investigated property sheets and
system settings, combed through the Windows registry, searched
through Microsoft's forums, searched through the GNU/Linux forums,
looked through numerous other forums with information on Vista, and
solicited advice from visitors to this site.
The only workaround I have found (other than abandoning the Windows
environment altogether) is to port the code to a compiler and
development environment which links to the Win32 API. There are
several of these available for download without charge.
After months of experimentation, I have settled on
MinGW/MSYS as the best
development environment for my purposes. Its advantages include the
following:
- The fastest executables of any environment tested.
- Superior portability of executables---generally speaking, no external
DLLs are required, other than the Windows system DLLs (but see the
note below). However, the static executables are
a good deal bulkier than those produced by many other compilers.
- The executables conform to the Win32 API, eliminating the 32MB
memory limitation and other Windows compatibility problems.
- A GNU/Linux-like MSYS environment for configuring, making, building,
and installing packages (such as GMP
and MPFR).
- Portability of the MinGW/MSYS installation among systems; MinGW and
MSYS are not tangled with the Windows Registry, so MinGW/MSYS can
be migrated to another Windows machine by simply copying the MinGW
and MSYS directories to the new machine. Of course, if the new
machine has a different class of processor (e.g., Pentium-4 as
opposed to Celeron), you may need to rebuild some packages.
- Full support (for internal calculations) of 80-bit long doubles
(including transcendentals) and 64-bit integers; note, however, that
the printf/scanf family produces erroneous results for long doubles,
and uses peculiar syntax for 64-bit integers.
Disadvantages of the MinGW/MSYS environment include the following:
- Several of the function calls are linked to the Microsoft Visual C++
run-time library MSVCRT.DLL, which explains some of the shortcomings
mentioned below. This also means that the behavior of the
executables is to some extent dependent on the version of Windows.
- There is only minimal support for the "conio" functions
(gotoxy, wherex, wherey, clrscr, etc.) familiar to users of
DJGPP and Borland C. However, conio
simulation libraries are available for MinGW and other
conio-challenged operating systems; these provide some, but not
all the functionality of DJGPP/Borland conio.
- The printf/scanf family of functions is unable to correctly handle
the 80-bit long double floating point variables native to the x86
Intel processors and FPUs (and compatible processors, such as AMD).
Either the value must be cast to double and printed, with resulting
loss of precision (16 digits rather than 19), or a custom function
must be employed for the purpose. I use the custom functions
___strtold and __szLD from my own trn library,
as in the following code fragment:
#include "trn.h"
...
char sz[256];
long double ld=3.1415926535897932384626433832795L;
...
printf("\n ld=%s\n", __szLD(sz, ld, "%.19Lf"));
...
The code must be compiled and linked with trn.c and the GMP library.
Alternatively, as several respondents have pointed out, third-party
substitutes for printf/scanf with support for long doubles are
available for MinGW.
- The printf/scanf family of functions expects the non-standard length
specifier "I64" for 64-bit integers ("ll" and
"i64" may not work, depending on the Windows version).
- The executables may behave differently when run within the MSYS
environment, as opposed to the Windows command-line box. I use
the MSYS environment primarily for compiling and linking codes (as
well as building and installing packages such as GMP and MPFR), then
run the executables within a Windows command-prompt box.
ADDITIONAL NOTES
- On 10/11 April 2007, I received an e-mail purporting to be from an
employee of Microsoft, or one of its contractors or
subcontractors. Following is the crux of the e-mail.
...I work for 16 side of OS (NTVDM). First of all I would like to thank you
for pointing out a bug in 16 bit Memory allocation. To give some background,
this was not a regression. This was done to restrict some apps from gulping
huge memory in WIN2K3 timeframe, but nobody objected then so it continued.
Now that people are asking for this, We are planning to fix this in SP1 of
VISTA.
Your inputs are very valuable to us, Help us to help you.
Thanks,
Subhash (Badri)
Judging from the headers, it was sent from southeast Asia, perhaps
Singapore, Taiwan, Malaysia, the Phillipines, Indonesia, Thailand,
or Vietnam. I cannot guarantee the bona fides of the e-mail; however,
based on internal consistency, it appears to be legitimate. If so,
the release of Vista SP1 will be of special interest.
- Vaclav Kotesovec reports that he installed the Vista SP1 Release
Candidate on 9 February 2008, and that the 32MB limitation is still
present. Daniel C. Marinescu reports that the final release of SP1
is expected prior to 30 June 2008. On 22 February 2008, the
BBC
reported that the final release of SP1 was expected in mid-March
of 2008, and that Microsoft indicated it might "break"
a number of previously functional applications, many of them anti-virus
and security oriented.
-
Microsoft's C compilers no longer support 80-bit long doubles; see
http://msdn2.microsoft.com/en-us/library/9cx8xs15(vs.80).aspx)
(thanks to William S. for providing this pointer). Long doubles
and doubles have the same representation (53-bit mantissa), with no
data type for the x87 80-bit long doubles. MSVC has no native support
for GMP or MPFR, although a procedure (which I have not tested) for
building GMP under MSVC is available at
http://fp.gladman.plus.com/computing/gmp4win.htm; thanks to
Andy Cadley for providing this pointer. MSVC has only token support
for DJGPP/Borland style conio functions.
- A significant disadvantage of the
Cygwin environment is the
necessity for some of the product's DLLs (such as cygwin1.dll
and cyggmp-3.dll) to be present on the end-user's system.
Compiling with the -mno-cygwin switch can produce a self-contained
executable (essentially a MinGW executable, but with slightly
different properties). Also, glibc is not available in Cygwin,
only the more limited Red Hat alternative "newlib".
Cygwin has very little support for conio, and the "system"
function is inoperative when a Cygwin executable is run in a Windows
command box (Cygwin looks for a non-existent command processor
/bin/sh). Cygwin lacks long double transcendental functions.
- Borland C
5.51 lacks any support for GMP or MPFR. Executable performance
is noticeably slower than that of most other compilers. Also, the
interface to 64-bit integers does not conform to either
the C99:TC2 standard or the C++0x working
draft N2135 (6 November 2006). Neither "long long"
nor "int64_t" is recognized, and these declaration
specifiers are part of both
the C99:TC2 standard and the C++0x
working draft N2135 (6 November 2006). For example, see pages 100
and 256 of
ISO/IEC 9899:TC2, document WG14_N1124, and pages 129 and 408 of
the C++0x working draft,
document N2135, ISO/IEC JTC 1/SC22/WG21 (6 November 2006),
Furthermore, the downloadable compiler package lacks the assembler
(TASM32), and thus inline assembly is unavailable. Documentation is
very skimpy. Finally, it is not clear from the license accompanying
the download whether or not free distribution of source and executables
is permitted, in compliance with the
GNU GPL.
- I have observed the following numerical bugs in Borland C 5.51.
- Significant digits beyond the 19th in long doubles are not
reliably processed in I/O, even when the number is exactly
representable as a long double. Thus printf("%.1Lf", ld), where
ld=11e18L + 1.0L, actually prints out "11000000000000000000.0".
- Implicitly or explicitly casting an integer-valued long double
to an unsigned long long (or unsigned __int64, as Borland labels it)
may lock up the code if the value of the long double exceeds
2^63 - 1 ~ 9.223372e18, even though both long doubles and ULLs can
represent exactly all positive integers less than
2^64 ~ 1.8446744e19.
- Preliminary evaluation of the
Digital Mars
compiler indicates that this product has quite extensive capabilities;
it can produce Win32 GUI, Win32 console, or 32-bit extended DOS
executables (and even the venerable 16-bit DOS .COM executables). A
notable shortcoming is the lack of any support for GMP or MPFR; as with
Borland C, porting GMP and MPFR looks like a major project. Support for
conio is extensive, but the interface is quite different from
Borland/DJGPP. The documentation format is inconvenient.
- I have observed the following numerical bugs in the Digital Mars
8.49 compiler.
- Significant digits beyond the 18th in long doubles are not
reliably processed in I/O, even when the number is exactly
representable as a long double. Thus printf("%.1Lf", ld),
where ld=4e18L + 746.0L, actually prints out "4000000000000000745.4"
(apparently a loss of two bits of precision).
- Implicitly or explicitly casting an integer-valued long double
to an unsigned long long may result in loss of precision if the
long double value exceeds 2^53 ~ 9.0072e15. Thus if
ld=1.82e19 + 1024.0L, the assignment statement ull=ld will in
fact produce a value of ull=18200000000000000000---a loss of 11
bits of precision. It is as if the long double is being cast to
a double before being cast to an unsigned long long.
- Static global arrays of long integers, used to store multiple
precision BIGINT integers, generate erroneous (arithmetical) results
(other compilers do not exhibit this problem when compiled from
the same source). If these variables are declared globally without
the static attribute, the errors disappear.
- The executables produced by the Digital Mars compiler once again
illustrate the Vista memory problem. If the Win32 compiler is
applied to vista1.c (as the default within Windows, or using the
-mn switch), the resulting executable is linked to the Win32 API
and does not exhibit the 32MB memory limit in Vista. However, if a
32-bit DOS executable is generated (using the -mx switch and
linking with X32.LIB, containing Doug Huffman's X-32 DOS
extender), it does not link to the Win32 API. The
32-bit DOS executable is then subject to Vista's 32MB memory
limitation, even though X-32 defers to the Windows DPMI server.
As with DJGPP's GCC executable, it is not subject to this
limitation in XP, Win98SE, or standalone DOS.
- Illka Blom reports that his code
Alybadix, designed
to solve chess problems quickly and compiled under DJGPP, also
encounters the 32MB memory limitation under Vista, even though
it succeeds in allocating up to 2000MB of memory under Windows XP
and DOS.
- DOSBox 0.70 from
SourceForge.net, installed
under Vista, is not subject to the 32MB memory limitation,
apparently because it links to the Win32 API. It does have a
self-imposed default memory limit of 16MB, which can be increased
to 64MB by editing the configuration file dosbox.conf; to go
beyond 64MB you would need to modify and recompile the source.
Thanks to Dos Freak and Daniel Verkamp for this information.
- Christopher C. Perry of Michigan State University suggested copying
the cmd.exe command interpreter from XP into Vista to see if it
would (1) start up, and (2) solve the memory problem. It does
start up and function, but is subject to the same memory limitation
as Vista's own cmd.exe. It also is unable to operate in full screen
mode within Vista, although it does so in Windows XP. Copies of
cmd.exe from both the 2001 and SP2 versions of XP were tried. In
addition, the versions of command.com from Vista, XP SP2, and Win98SE
were tried; none of them were functional within Vista.
- Iulian-Nicu Serbanoiu suggested that the 32MB limitation might apply
only to individual arrays, so that it might be possible to allocate
a much larger amount of total memory using several smaller chunks.
However, testing confirms that Vista is allocating a maximum of 32MB
to the entire process, so that the aggregate memory allocated to
several different arrays still cannot exceed 32MB. Serbanoiu also
points out that a special external DLL (mingwm10.dll) is required for
MinGW code which uses thread-safe exception handling.
- The Vista command box is incapable of running in full-screen mode,
rendering it incompatible with many older applications (particularly
games).
- The Vista command box has been observed to commit errors in
clearing the (console) screen and restoring the cursor.
- Vista's command prompt loads by default a version of HIMEM.SYS (the
same one present in XP), and is only marginally functional
without it.
- Vista as shipped is unable to read Windows .HLP help files
written for previous versions of Windows or their applications.
In fact, Vista is unable to read its own
.HLP help files (those shipped with the OS distribution). A fix
is available on Microsoft's site; however, Mark Odell has provided
a pointer to a Microsoft-supplied
standalone update installer which will fix the problem.
- Nero Express 7 (OEM, Burning ROM version 7.2.3.2) will not install
in Vista. Incompatibility warnings appear at the beginning of
installation; if ignored, the installation will crash at an
intermediate stage. The application was purchased 12 March 2007 at
Best Buy. An update/fix may (or may not) be available for download
from Nero, depending
on your specific product version; if so, anticipate that it may be
on the order of 100MB in size.
- For those interested in downloading and installing the latest
DJGPP 2.04 beta files, see
djgpp204.html.