Issue has been resolved. What we ultimately concluded was that for unknown reasons the order of files being required was slightly different for different users of the docker container.So the solution we have found was to explicitly re-order requires and to require everything needed for a given module, even if it was typically already loaded elsewhere. Luckily ruby deals with redundant dependency loading for me.We never found a reason why some users, repeatably and without fail, experienced different behavior when loading the application.-Hi Everyone,I've just spent two days dealing with some weirdness, and I turn to you all to see if anyone else has had an experience like this.Background on me: I'm not a docker newbie. I've been working with docker professionally for 3+ years now, and I was playing with it with amateur interest from before DockerFiles existed.I built a docker container that we are using for development, pushed it to our repo, and had a colleague pull it and try to use it.The container works fine for me, and for my colleague it gets a dependency missing error. We then tried it across other colleagues. For some it works fine, for some more it does not.We confirmed the container is the same, we have the same SHA for the image digest when we run `docker images -digest`.What we have confirmed is that for those that it does not work, they are using OSX High Sierra, where as for myself and another colleague for whom it does work, we are on (regular) Sierra.So, thats where I am now. We think the difference is caused by OSX version, but I don't know how to mitigate it, or how to further diagnose the issue.Do you have any thoughts?
![]()
So we began research to optimize performance on Mac and ease our. If you're still on.qcow2 then upgrade to.raw if system disk filesystem is APFS. Ip: '192.168.111.16' config.vm.provider:virtualbox do v v.customize. Docker For Mac has really changed how I work: I now use it for all my linux-related developments. The integration is OS X is really well done and it’s really perfect for a development environment. The only problem is that Docker For Mac uses a file called Docker.qcow2 that takes more and more disk space as time passes (mine got to 20GB).
Any ideas of where to look to see a root cause?. More than likely you and your colleague's docker is using different Linux kernel (with possible differences in the base OS, not to mention different version of docker).Docker for Mac uses XHYVE hypervisor and runs (at least in my version) Linuxkit base OS.You.CAN. 'enter' that Linux VM on a mac and verify a few possible differences this way:screen /Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/ttyThis will put you into a VM shell, you may be able to run a few checks there and compare.
This will tell the story. There were a few devs on my team that were tracking down similar weirdness with Docker for Mac. I don't recall the details and wasn't directly involved, but they did have problems that were, in part, traced to some edge/experimental stuff that some had turned on and others didn't. In the end, they spent a few hours huddled together with lsof, strace and friends and the answer was found. I believe it turned out to be PEBKAC as opposed to anything specifically blameable to Docker. Since you mentioned that some developers are on Sierra and others on High Sierra, I'm guessing that the behaviour you're seeing might relate to an issue with raw images on APFS in High Sierra. For Sierra users, qcow2 images are used.If you look in Docker → Preferences → Disk, you can figure out disk format by the file extension on the Disk image location.
See for a bit of information about the two image formats.To rule this out as an issue, I suggest you manually switch everyone to the qcow2 image format, and try again. Future versions of Docker for Mac will switch to the raw format as a default. Sorry, I should have been more clear. I'm referring to the VM image that backs the xhyve VM that Docker for Mac uses, not the Docker images. A problem cropped up many months ago when Docker for Mac started switching High Sierra users from qcow2 to raw VM images. This transition has been temporarily halted.
But, depending on when your High Sierra users installed Docker for Mac, they may be on raw VM images.I suggest you check your High Sierra users for what type of VM image they're on, and switch them all to qcow2, if they're on raw. Here's how to do this. Stop Docker for Mac. Edit /Library/Group Containers/group.com.docker/settings.json, and change the file extension on diskPath from raw to qcow2.
Start Docker for Mac.
Expected behaviorDocker.raw should not grow much larger than the space used by docker containers, images, volumes. Actual behaviorI have very few images downloaded locally, with no active containers and just a two volumes with some data. ➜ com.docker.driver.amd64-linux docker system dfTYPE TOTAL ACTIVE SIZE RECLAIMABLEImages 19 0 3.628GB 3.628GB (100%)Containers 0 0 0B 0BLocal Volumes 2 0 803.5MB 803.5MB (100%)Build Cache 0B 0BNevertheless Docker.raw size is huge! ➜ com.docker.driver.amd64-linux ls -l Docker.raw-rw-r-r-@ 1 cf staff 6 Dec 7 10:35 Docker.rawInformationDiagnose info: Docker for Mac: version: 17.11.0-ce-mac40 (b2149617ed8d7263c9c035e25fe48c)macOS: version 10.13.1 (build: 17B1003)logs: /tmp/F397F806-6CA7-40E3-9C36-79B5F0ED519D/20108.tar.gzOK db.gitOK vmnetdOK dnsOK driver.amd64-linuxOK virtualization VT-XOK appOK mobyOK systemOK moby-syslogOK kubernetesOK dbOK envOK virtualization kern.hvsupportOK slirpOK osxfsOK moby-consoleOK logsOK docker-cliOK menubarOK disk. I also tried to do a full system prune ➜ com.docker.driver.amd64-linux docker volume pruneWARNING! This will remove all volumes not used by at least one container.Are you sure you want to continue?
![]() ![]()
y/N yDeleted Volumes:033ce3294aa395bbb8c468eba823622d65b5b1d7f042c4a7ebf29826aed6ea402901af7dd8d50ae74ad761fe33cc558881d46ab323e7f6ee6eb87c3337ed8508Total reclaimed space: 803.5MB➜ com.docker.driver.amd64-linux docker system prune -aWARNING! This will remove:- all stopped containers- all networks not used by at least one container- all images without at least one container associated to them- all build cacheAre you sure you want to continue? y/N yDeleted Images:untagged: postgres:9.5.4untagged: postgres@sha256:1480f2446dabb1116fafa426ac5873a84dc4a4d0d9d4b37a5601e8deleted: sha256:2417ea518abc0db32cf2fb0d021ce57dab2e16f480ac924000e52afd07d3b0a4.deleted: sha256:b51149973e6a6c4fb1091ef34ff70002ee307e9cf226004a93c9b7Total reclaimed space: 3.628GBNevertheless Docker.raw size didn't shrink even after restarting docker engine. Ls -l shows the 'size' of the file, not the number of sectors allocated by the filesystem.
Try ls -sl: $ cd /Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux$ ls -sl Docker.raw16248952 -rw-r-r-@ 1 user staff ` Dec 7 13:57 Docker.rawIn my case Docker.raw is only using 16248952 filesystem sectors which is much less than the maximum file size of 6 bytes. If I download some images then the 16248952 increases, and if I docker system prune then it decreases again.I think this is working as expected so I'll close the ticket but nevertheless it is a little confusing. Perhaps we could present the current space usage in the UI somehow- I'll escalate this suggestion to our UI team.Thanks for the report and for using Docker for Mac! Just updated to 17.12.0-ce-mac47 on 10.13.2 on a 13' MBA, reset the preferences and deleted all the things.Preferences say the virtual disk image size is 64GB(3.5GB allocated)Previously, my.qcow2 file was 6.5GBNow ls -sl Docker.raw reads 6840456 -rw-r-r-@ 1 user staff 6 Jan 14 11:03 Docker.raw - 64GB. When I use GrandPerspective, it also reports that the.raw file is 64GB in size, when it used to report that the qcow2 was 6.5GB and it had some images in it, not a fresh install. Looks like there might still be a problem here. I was using rsync to do simple backup of my home directory when I discovered the humongous size of Docker.raw.
I was able to solve my problem by moving the file to /scratch under Preferences Disk. It also had an option to switch the size from 64GB to 32GB, which is better, but still very large. I'd rather not backup GBs of zeros or else loose data I actually do want.Since the file doesn't actually use disk space until it is 'filled', I guess this is fine. But additional options for smaller file sizes might still be beneficial in certain situations. All in all I even doubt that the benefit of preallocating all that space is worth the weird side effects and head aches many user will experience because of it. Keep in mind that many not-very-technical users are instructed to install docker for a variety of reasons. After you delete a large file from inside the Linux VM, will the space be reclaimed from the Docker.raw file?
From I'm suspecting the answer is no. My suspicion is that Linux does it's normal filesystem delete where it just unlinks the inode reference, but the underlying sectors on the disk are not zeroed out. And without those filesystem bits being zeroed out, APFS will never reclaim the sparse bits of the file. Is there a better option to reclaim that disk space other than wiping the Docker.raw file and starting fresh? I am experiencing the same issue but it is causing issues.$ uname -rmsDarwin 17.7.0 x8664$ docker -versionDocker version 18.06.1-ce, build e68fc7aWhen I try to run my postgres container I am told that I don't have enough disk space on the device. I don't understand how come this issue has been closed.This is clearly a big problem, as Docker for Mac is eating up all disk space.
The fact that the space may be just 'virtually' used doesn't change a thing from the user's point of view.In the FAQs , it says:'This is an illusion. The output of ls is misleading, because it lists the logical size of the file rather than its physical size.'
It may as well be an illusion, but I can't install software updates and MacOS is constantly complaining about running out of space.Kindly reconsider opening this issue, as it's a very, very annoying thing that imo should not happen in stable software. I understand it may not be easy to fix, but this is Docker for Mac at the end of the day - saying that it's kind of MacOs fault is a bit of a stretch, at best.Thanks.Edit: pingWould you please consider at least reopening this issue?Regarding: 'I think this is working as expected': people here have their macbooks unusable because of this disk space issue.
![]() Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
March 2023
Categories |