Can now enable zip file system read/write cache. This will copy files that are read or written to out of the zip and to the host file system. When not enabled, only file writes are copied from the zip file system to the host.

This commit is contained in:
James Bryant
2025-03-28 17:04:22 -07:00
parent dde20d6cf5
commit 572ca8e38e
18 changed files with 123 additions and 48 deletions

10
Jenkinsfile vendored
View File

@@ -31,6 +31,7 @@ pipeline {
sh '''#!/bin/bash
source ~/emsdk/emsdk_env.sh
cd project/emscripten
make clean
make test
killall -9 python3
cd Build/Test
@@ -48,6 +49,7 @@ pipeline {
}
dir("project/linux") {
sh '''#!/bin/bash
make clean
make test || exit
./Build/Test/boxedwine || exit
make testMultiThreaded || exit
@@ -81,6 +83,7 @@ pipeline {
}
dir("project/linux") {
sh '''#!/bin/bash
make clean
make testMultiThreaded || exit
./Build/TestMultiThreaded/boxedwine
'''
@@ -130,7 +133,8 @@ pipeline {
source ~/emsdk/emsdk_env.sh
set -x
rm -rf Deploy
make
make clean
make multiThreaded
if [ ! -f "Build/Release/boxedwine.wasm" ]
then
echo "boxedwine.wasm DOES NOT exists."
@@ -163,6 +167,7 @@ pipeline {
rm Build/MultiThreaded/boxedwine
rm Build/Release/boxedwine
rm Build/Deploy/Linux64/boxedwine
make clean
make release
make multiThreaded
mkdir -p Build/Deploy/Linux64
@@ -226,8 +231,9 @@ pipeline {
dir("project/linux") {
sh '''#!/bin/bash
rm Build/MultiThreaded/boxedwine
rm Deploy/LinuxArm64/boxedwine
rm Deploy/LinuxArm64/boxedwine
mkdir -p Deploy/LinuxArm64
make clean
make multiThreaded
if [ ! -f "Build/MultiThreaded/boxedwine" ]
then

View File

@@ -19,6 +19,8 @@ This will load the file system from wine.zip and write the changes to c:\root
-bpp X : Value must be 16 or 32. Most games are fine with 32-bit.
-cacheReads : will copy files from zip file system to host local file system when reading a file. By default only files opened for write get cached.
-cpuAffinity X : For multi-threaded builds, this will set the CPU affinity for the app/game. Normally you should just pass in 1 if this is needed. Some older games that use multiple threads sometimes require this.
-ddrawOverride path : will enabled CNC DDraw wrapper for the path passed in. The path needs to be the full emulated file system path, for example, /home/username/.wine/drive_c/mdkperf/PERF_W95.EXE

View File

@@ -38,13 +38,14 @@ If your application needs to be installed and sets registry entries or installs
3. Decide which file system you want to use.
a. You can use boxedwine.zip that is used in the ski32 demo
b. Or you can use a full file system of your choosing.
The pros of using boxedwine.zip is that you will have a small file system. The cons are that this small file system had some files removed in order to make it that small, so not all applications will work with it. But a lot of applications and games should work with it just fine. All the Demos in the Boxedwine UI work with it.
I'd recommend trying boxedwine.zip unless you know it won't work for you.
4. If you want to use a full file system continue with the following steps, but substitute boxedwine.zip with your full file system of choice or rename that file system boxedwine.zip.
4. If you want to use a full file system continue with the following steps, but substitute boxedwine.zip with your full file system of choice or rename that file system to boxedwine.zip.
5. You need to copy boxedwine.zip to the FileSystems2 folder in the Boxedwine data folder, you can find the location of the data folder in the Boxedwine UI under the Options tab. On Windows this is "%APPDATA%\Boxedwine"
@@ -56,7 +57,7 @@ If your application needs to be installed and sets registry entries or installs
9. Now go to the Container that has your installed application. You can find this folder in the Boxedwine UI by going to the Containers tab and selecting your app on the left. On the right hand side you will see "Storage Location", if you hit the "Open" Boxedwine should take your computer to that folder. In that folder open "root". There should be a subfolder called "home" there. Using whatever zip tools you prefer, like 7zip, copy the home folder to boxedwine.zip, this will merge the new home folder contents with the existing home folder in boxedwine.zip.
10. You now have a boxedwine.zip file system that contains your installed game and you should be able to run this on the web.
10. You now have a boxedwine.zip file system that contains your installed application and you should be able to run this on the web.
When it comes to running the application you just installed, probably the easiest way will be to create a bat file in home/username directory of boxedwine.zip. For example, if you followed the above steps with the Caesar 3 demo, your games is installed in home\username\.wine\drive_c\SIERRA\CAESAR3DEMO inside boxedwine.zip. A suitable bat file to run this will look like

View File

@@ -1,21 +1,14 @@
### In Progress
1. Improve performance on Raspberry Pi for 64-bit OS
2. Make official build for Mac Intel/Arm
Update (12/30/2023)
1. Armv8 on Raspberry Pi is working well, just a couple bugs left, mainly around 16-bit apps
2. Mac Intel builds are available on the build server (click green check next to checkin, then click "details", then click artifacts on top right of Jenkins)
3. Mac Arm still has some issues, mainly around the 16k page size and trying to emulate a 4k page size
1. Improve performance for Emscripten/Web build
2. Reduce memory usage
### Sometime in the distant future
1. Add support to mount ISO's
2. Gecko support
3. .NET and Java support
4. Copy/Paste support
5. Improve performance for Emscripten/Web build
3. Steam Support
4. .NET and Java support
5. Copy/Paste support (DONE)
6. Add the ability to launch Dosbox for the few Windows games that use Dos installers.
7. Joystick support using SDL
8. Mouse Capture support
9. Add Android support

View File

@@ -103,6 +103,7 @@ public:
static BString exePath;
static bool disableHideCursor;
static bool forceRelativeMouse;
static bool cacheReads;
static bool useF64;
static void init();

View File

@@ -167,8 +167,10 @@ U64 FsFileNode::length() {
}
BString FsFileNode::getLink() {
#if defined(BOXEDWINE_ZLIB) && !defined(__EMSCRIPTEN__) && defined(BOXED_MOVE_TOUCHED_ZIP_FILES_TO_HOST)
ensurePathIsLocal(false);
#if defined(BOXEDWINE_ZLIB) && !defined(__EMSCRIPTEN__)
if (KSystem::cacheReads) {
ensurePathIsLocal(false);
}
#endif
return this->link;
}
@@ -207,8 +209,8 @@ FsOpenNode* FsFileNode::open(U32 flags) {
if ((flags & K_O_ACCMODE)==K_O_RDONLY) {
openFlags|=O_RDONLY;
// make the file local so that it will load faster (no inflate and weird seeking logic)
#if defined(BOXEDWINE_ZLIB) && !defined(__EMSCRIPTEN__) && defined(BOXED_MOVE_TOUCHED_ZIP_FILES_TO_HOST)
if (this->zipNode) {
#if defined(BOXEDWINE_ZLIB) && !defined(__EMSCRIPTEN__)
if (KSystem::cacheReads && this->zipNode) {
ensurePathIsLocal(false);
}
#endif

View File

@@ -248,60 +248,92 @@ S32 translateNativeSocketError(const std::shared_ptr<KNativeSocketObject>& s, in
if (error == WSAENOTCONN) {
result = -K_ENOTCONN;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ENOTCONN", result);
} else if (error == WSAEWOULDBLOCK) {
}
else if (error == WSAEWOULDBLOCK) {
result = -K_EWOULDBLOCK;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EWOULDBLOCK", result);
} else if (error == WSAETIMEDOUT) {
}
else if (error == WSAETIMEDOUT) {
result = -K_ETIMEDOUT;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ETIMEDOUT", result);
} else if (error == WSAECONNRESET) {
}
else if (error == WSAECONNRESET) {
result = -K_ECONNRESET;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ECONNRESET", result);
} else if (error == WSAEDESTADDRREQ) {
}
else if (error == WSAEDESTADDRREQ) {
result = -K_EDESTADDRREQ;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EDESTADDRREQ", result);
} else if (error == WSAEHOSTUNREACH) {
}
else if (error == WSAEHOSTUNREACH) {
result = -K_EHOSTUNREACH;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EHOSTUNREACH", result);
} else if (error == WSAECONNREFUSED) {
}
else if (error == WSAECONNREFUSED) {
result = -K_ECONNREFUSED;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ECONNREFUSED", result);
} else if (error == WSAEISCONN) {
}
else if (error == WSAEISCONN) {
result = -K_EISCONN;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ECONNREFUSED", result);
} else if (error == WSAEMSGSIZE) {
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EISCONN", result);
}
else if (error == WSAEMSGSIZE) {
result = -K_EMSGSIZE;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EMSGSIZE", result);
} else {
}
else {
result = -K_EIO;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EIO", result);
}
#else
if (error == ENOTCONN)
if (error == ENOTCONN) {
result = -K_ENOTCONN;
else if (error == EWOULDBLOCK)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ENOTCONN", error);
}
else if (error == EWOULDBLOCK) {
result = -K_EWOULDBLOCK;
else if (error == ETIMEDOUT)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EWOULDBLOCK", error);
}
else if (error == ETIMEDOUT) {
result = -K_ETIMEDOUT;
else if (error == ECONNRESET)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ETIMEDOUT", error);
}
else if (error == ECONNRESET) {
result = -K_ECONNRESET;
else if (error == EDESTADDRREQ)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ECONNRESET", error);
}
else if (error == EDESTADDRREQ) {
result = -K_EDESTADDRREQ;
else if (error == EHOSTUNREACH)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EDESTADDRREQ", error);
}
else if (error == EHOSTUNREACH) {
result = -K_EHOSTUNREACH;
else if (error == EISCONN)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EHOSTUNREACH", error);
}
else if (error == EISCONN) {
result = -K_EISCONN;
else if (error == ECONNREFUSED)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EISCONN", error);
}
else if (error == ECONNREFUSED) {
result = -K_ECONNREFUSED;
else if (error == EMSGSIZE)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "ECONNREFUSED", error);
} else if (error == EMSGSIZE) {
result = -K_EMSGSIZE;
else if (error == EAFNOSUPPORT)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EMSGSIZE", error);
} else if (error == EAFNOSUPPORT) {
result = -K_EAFNOSUPPORT;
else if (error == EINPROGRESS)
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EAFNOSUPPORT", error);
} else if (error == EINPROGRESS) {
result = -K_EINPROGRESS;
else
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EINPROGRESS", error);
} else if (error == EPERM) {
result = -K_EPERM;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "EPERM", error);
} else {
result = -K_EIO;
LOG_SOCK(" native socket: %x error %s(%x)", s->nativeSocket, "UKNOWN/EIO", error);
}
#endif
return result;
}
@@ -383,6 +415,7 @@ KNativeSocketObject::KNativeSocketObject(U32 domain, U32 type, U32 protocol) : K
nativeProtocol = IPPROTO_IP;
}
this->nativeSocket = (S32)socket(AF_INET, nativeType, nativeProtocol);
LOG_SOCK("%x native socket: %x open nativeType=%x nativeProtocol=%x", KThread::currentThread()->id, nativeSocket, nativeType, nativeProtocol);
#ifndef BOXEDWINE_MULTI_THREADED
setNativeBlocking(this->nativeSocket, false);
#endif

View File

@@ -61,6 +61,7 @@ std::shared_ptr<FsNode> KSystem::procNode;
U32 KSystem::wineMajorVersion;
bool KSystem::disableHideCursor = false;
bool KSystem::forceRelativeMouse = false;
bool KSystem::cacheReads = false;
#if defined(BOXEDWINE_X64) && !defined(BOXEDWINE_USE_SSE_FOR_FPU)
bool KSystem::useF64 = false;

View File

@@ -266,6 +266,9 @@ std::vector<BString> StartUpArgs::buildArgs() {
if (this->forceRelativeMouse) {
args.push_back(B("-forceRelativeMouse"));
}
if (this->cacheReads) {
args.push_back(B("-cacheReads"));
}
for (auto& a : this->args) {
args.push_back(a);
}
@@ -282,6 +285,7 @@ bool StartUpArgs::apply() {
#endif
KSystem::disableHideCursor = this->disableHideCursor;
KSystem::forceRelativeMouse = this->forceRelativeMouse;
KSystem::cacheReads = this->cacheReads;
KSystem::pentiumLevel = this->pentiumLevel;
KSystem::pollRate = this->pollRate;
if (KSystem::pollRate < 0) {
@@ -801,7 +805,10 @@ bool StartUpArgs::parseStartupArgs(int argc, const char **argv) {
this->disableHideCursor = true;
} else if (!strcmp(argv[i], "-forceRelativeMouse")) {
this->forceRelativeMouse = true;
} else if (!strcmp(argv[i], "-dxvk")) {
} else if (!strcmp(argv[i], "-cacheReads")) {
this->cacheReads = true;
}
else if (!strcmp(argv[i], "-dxvk")) {
BString dxvk;
dxvk = argv[i + 1];
this->enableDXVK = dxvk.startsWith('t', true) || (dxvk == "1") || dxvk.startsWith('y', true);

View File

@@ -114,6 +114,7 @@ public:
bool enableDXVK = false;
bool disableHideCursor = false;
bool forceRelativeMouse = false;
bool cacheReads = false;
private:
bool workingDirSet = false;

View File

@@ -192,6 +192,12 @@ void OptionsView::createGeneralTab() {
};
#endif
cacheFilesControl = section->addCheckbox(Msg::OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_LABEL, Msg::OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_HELP, GlobalSettings::enabledCachedReadFiles);
cacheFilesControl->onChange = [this]() {
GlobalSettings::enabledCachedReadFiles = this->cacheFilesControl->isChecked();
GlobalSettings::saveConfig();
};
std::shared_ptr<LayoutSection> bottomSection = model->addSection();
bottomSection->addSeparator();
BString deleteLabel;

View File

@@ -57,6 +57,7 @@ private:
std::shared_ptr<LayoutComboboxControl> scaleControl;
std::shared_ptr<LayoutComboboxControl> openGlControl;
std::shared_ptr<LayoutCheckboxControl> automationControl;
std::shared_ptr<LayoutCheckboxControl> cacheFilesControl;
// Display
std::shared_ptr<LayoutComboboxControl> themeControl;

View File

@@ -154,6 +154,7 @@ void BoxedContainer::launch(const std::vector<BString>& args, BString labelForWa
GlobalSettings::startUpArgs.setScale(GlobalSettings::getDefaultScale());
GlobalSettings::startUpArgs.setVsync(GlobalSettings::getDefaultVsync());
GlobalSettings::startUpArgs.setResolution(GlobalSettings::getDefaultResolution());
GlobalSettings::startUpArgs.cacheReads = GlobalSettings::cacheReads();
this->launch();
GlobalSettings::startUpArgs.addArgs(args);
GlobalSettings::startUpArgs.readyToLaunch = true;
@@ -168,6 +169,7 @@ void BoxedContainer::launch() {
GlobalSettings::startUpArgs.setScale(GlobalSettings::getDefaultScale());
GlobalSettings::startUpArgs.setVsync(GlobalSettings::getDefaultVsync());
GlobalSettings::startUpArgs.setResolution(GlobalSettings::getDefaultResolution());
GlobalSettings::startUpArgs.cacheReads = GlobalSettings::cacheReads();
std::shared_ptr<FileSystemZip> fs = this->fileSystem.lock();
GlobalSettings::startUpArgs.addZip(fs->filePath);
BString root = GlobalSettings::getRootFolder(this);

View File

@@ -77,6 +77,7 @@ int GlobalSettings::lastScreenX;
int GlobalSettings::lastScreenY;
U32 GlobalSettings::defaultOpenGL = OPENGL_TYPE_DEFAULT;
bool GlobalSettings::enabledAutomation = false;
bool GlobalSettings::enabledCachedReadFiles = false;
void GlobalSettings::init(int argc, const char **argv) {
GlobalSettings::largeFontBold = nullptr;
@@ -124,6 +125,7 @@ void GlobalSettings::init(int argc, const char **argv) {
GlobalSettings::lastScreenY = config.readInt(B("WindowY"), 0);
GlobalSettings::defaultOpenGL = config.readInt(B("OpenGL"), OPENGL_TYPE_DEFAULT);
GlobalSettings::enabledAutomation = config.readBool(B("EnableAutomation"), false);
GlobalSettings::enabledCachedReadFiles = config.readBool(B("EnabledCachedReadFiles"), false);
if (!Fs::doesNativePathExist(configFilePath)) {
saveConfig();
@@ -199,6 +201,8 @@ void GlobalSettings::saveConfig() {
config.writeInt(B("WindowY"), GlobalSettings::lastScreenY);
config.writeInt(B("OpenGL"), GlobalSettings::defaultOpenGL);
config.writeBool(B("EnableAutomation"), GlobalSettings::enabledAutomation);
config.writeBool(B("EnabledCachedReadFiles"), GlobalSettings::enabledCachedReadFiles);
config.saveChanges();
}

View File

@@ -173,6 +173,7 @@ public:
static BString alternativeOpenGlLocation();
static BString alternativeOpenGlUrl();
static U32 alternativeOpenGlDownloadSizeMB();
static bool cacheReads() { return GlobalSettings::enabledCachedReadFiles; }
static StartUpArgs startUpArgs;
@@ -226,6 +227,7 @@ private:
static int lastScreenY;
static U32 defaultOpenGL;
static bool enabledAutomation;
static bool enabledCachedReadFiles;
};
#endif

View File

@@ -169,7 +169,11 @@ const char* c_getTranslation(Msg msg, bool useDefaultIfMissing) {
case Msg::OPTIONSVIEW_ENABLE_AUTOMATION_LABEL:
return "Enable Automation:";
case Msg::OPTIONSVIEW_ENABLE_AUTOMATION_HELP:
return "This option should not be enabled for most people. This will add an automation option to the shortcuts that will allow someone to record some acation and play them back for the app. This automation testing makes it easier to test apps quickly. It works by comparing screen shots so it is not very useful for screens that change with animations. F11 key will capture a full screen shot. If you want a partial screen shot, then hit the F11 key and keep it down. With the mouse hold down the left mouse button and raw a rectangle to capture just that area.";
return "This option should not be enabled for most people. This will add an automation option to the shortcuts that will allow someone to record some action and play them back for the app. This automation testing makes it easier to test apps quickly. It works by comparing screen shots so it is not very useful for screens that change with animations. F11 key will capture a full screen shot. If you want a partial screen shot, then hit the F11 key and keep it down. With the mouse hold down the left mouse button and raw a rectangle to capture just that area.";
case Msg::OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_LABEL:
return "Cache Opened Files:";
case Msg::OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_HELP:
return "When a file is opened by Boxedwine it will check the computers local file system first, and if its not there it will check the file system zip file. If the file is found in the zip, it will be read from the zip and left there. But if that file is written to, it will be copied to the computers local file system and opened from there. When Cached Open Files is checked, it will also copy files that are open for read only to the computers local file system. This might improve application start up time a tiny bit, but on a Windows host it will likely trigger anti virus or Microsoft Defender warnings about the Wine executables, like explorer.exe.\n\nThis option is also used for preparing an application and file system to be used on the Web with the Emscripten build. If you have a working zip file system that you want to use on the web, you can use this option to run your application and every file that the application opens will be cached in the computers local file system (root folder of the container). You can then zip up that the contents of that root folder to use as a smaller file system that has just the files needed for that application. If you do this you will probably want to excerise the application pretty good to make sure no files were missed.";
case Msg::OPTIONSVIEW_DEFAULT_FONT_SCALE_LABEL:
return "Font Scale";
case Msg::OPTIONSVIEW_DEFAULT_FONT_SCALE_HELP:

View File

@@ -175,6 +175,8 @@ enum class Msg {
OPTIONSVIEW_DEFAULT_OPENGL_HELP,
OPTIONSVIEW_ENABLE_AUTOMATION_LABEL,
OPTIONSVIEW_ENABLE_AUTOMATION_HELP,
OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_LABEL,
OPTIONSVIEW_ENABLE_COPY_TOUCHED_FILES_HELP,
OPTIONSVIEW_DEFAULT_FONT_SCALE_LABEL,
OPTIONSVIEW_DEFAULT_FONT_SCALE_HELP,
OPTIONS_VIEW_DELETE_ALL_BUTTON_LABEL,

View File

@@ -551,19 +551,23 @@ std::shared_ptr<U8[]> extractIconFromExe(BString nativeExePath, int size, int* w
if (!f.isOpen()) {
return nullptr;
}
unsigned char* buffer = new unsigned char[32*1024];
U32 read = (U32)f.read(buffer, 32 * 1024);
const int BUFFER_SIZE = 128 * 1024;
unsigned char* buffer = new unsigned char[BUFFER_SIZE];
U32 read = (U32)f.read(buffer, BUFFER_SIZE);
if (buffer[0]!='M' || buffer[1]!='Z') {
delete[] buffer;
return nullptr;
}
U32 nextHeader = READD(buffer, 60); // e_lfanew File address of new exe header
if (nextHeader>read-1) {
delete[] buffer;
return nullptr;
}
std::vector<IconInfo> icons;
if (buffer[nextHeader]=='N' && buffer[nextHeader+1]=='E') {
ImageOs2Header* header = (ImageOs2Header*)(buffer+nextHeader);
if (header->ne_rsrctab >= header->ne_restab) {
delete[] buffer;
return nullptr;
}
f.setPos(nextHeader + header->ne_rsrctab);
@@ -635,12 +639,13 @@ std::shared_ptr<U8[]> extractIconFromExe(BString nativeExePath, int size, int* w
if (resourceSize) {
f.setPos(rawOffset);
read = f.read(buffer, 32*1024);
readResourceDirectory(f, resourceRVA, rawOffset, buffer, 0, 32*1024, (ImageResourceDirectory*)buffer, 0, false, icons);
read = f.read(buffer, BUFFER_SIZE);
readResourceDirectory(f, resourceRVA, rawOffset, buffer, 0, BUFFER_SIZE, (ImageResourceDirectory*)buffer, 0, false, icons);
}
} else if (optionalHeader->Magic == 0x020b) {
kwarn("Icon 20b not implemented");
} else {
delete[] buffer;
return nullptr;
}
}
@@ -658,7 +663,9 @@ std::shared_ptr<U8[]> extractIconFromExe(BString nativeExePath, int size, int* w
bestIcon = icons[i];
}
}
delete[] buffer;
return parseIcon(f, bestIcon, width, height);
}
delete[] buffer;
return nullptr;
}