libarchive?

Hmm I'll take a look.

I need this to be an independent library as I need to port it to a arm platform that doesn't run a full version of Linux.

Well then I'm pretty sure it needs proper operating system, for filesystem access, mutexes, malloc and whatnot. Most libraries will make some assumptions

Yea I need something that can work only from memory.

The platform exposes a SDK which gives you access to the fs and stuff but it doesn't you the regular syscall names. I would have to load the files into memory before hand then apply the archive library to them.

If you need to kinda "preload" the files, can't you stick to one format to make it at least somewhat manageable? Then i'd probably just reimplement gzip as the simplest one (or maybe even evaluate if libz can be used?) Or maybe try one of the algorithms in busybox, code seems to be simple: https://github.com/mirror/busybox/tree/7fa799a97d381902ab27556918722a6e2d138b9e/archival


Note that if you're constrained in memory or cpu department, lzma or simillar may be completely unusable too

This post was edited by nuvo on May 16 2017 12:05pm

Dealing with compressed files in memory seems like it kind of defeats the purpose of compressing files in the first place.

How so?

You do realize windows compresses its own memory in memory.

That just blew my mind.

But 9 times out of 10 compressing files is just to save disk space or bandwidth. If you need work with the contents of a directory is there a reason you can just decompress it entirely first?

Directory size is one issue.

The reason for the memory constraint is because any library that handles loading of files itself will not work on the system. I would either have to load the entire archive into memory or some how wrap their loading calls around the systems sdk.

That's why I either want to manipulate the way the library reads the file so I can use the sdk read calls, or read the entire glob into memory which the library can do what ever it pleases with it.

Then again loading an entire file into memory may not be the best solution either.

Plus archives isn't only about compression.

Archiving is a way to store many files in a container. Compression is an after effect that most containers (except like tar) apply because it makes sense to do so.

Edit:: Anyways I decided to take a different approach. I'm going to make a web frontend that the application on the system can work with. This offloads the entire archive problem, memory, and file size problems to a remote standardized server while still having the functionality on the device.

This post was edited by AbDuCt on May 19 2017 10:44am

What's the device?