Trying to build the Hugo site textfiles.habd.as on a 2019 MBP and kept encountering the too many open file socket exception. Hugo seemed to have completed the build but was unable to subsequently serve the files:
Built in 110398 ms
Watching for changes in /Volumes/Inetpub/textfiles.bit/{archetypes,assets,content,data,la
youts,static,themes}
Watching for config changes in /Volumes/Inetpub/textfiles.bit/config/_default
Environment: "development"
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
ERROR 2020/11/09 14:11:00 Error: listen tcp 127.0.0.1:1313: socket: too many open files
Running ulimit -a I see the following output:
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-v: address space (kbytes) unlimited
-l: locked-in-memory size (kbytes) unlimited
-u: processes 2048
-n: file descriptors 524288
To rule out the likelihood of a possible regression in the installed Hugo version I did a quick brew upgrade hugo to download the latest hugo version, bumping the hugo binary from 0.74.3 up to 0.78.1 and rebuilt only to see:
Built in 199877 ms
Watching for changes in /Volumes/Inetpub/textfiles.bit/{archetypes,assets,content,data,layouts,static,themes}
Watching for config changes in /Volumes/Inetpub/textfiles.bit/config/_default
fatal error: pipe failed
And the following stack trace:
Expand to show stack trace
goroutine 1 [running]:
runtime.throw(0x5d079aa, 0xb)
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/panic.go:1116 +0x72 fp=0xc26a503858 sp=0xc26a503828 pc=0x403c212
runtime.sigNoteSetup(0x6e41c00)
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/os_darwin.go:98 +0xc5 fp=0xc26a503880 sp=0xc26a503858 pc=0x4038e65
os/signal.signal_enable(0x7ba4796400000002)
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/sigqueue.go:198 +0xa5 fp=0xc26a5038a0 sp=0xc26a503880 pc=0x4070b85
os/signal.enableSignal(...)
/usr/local/Cellar/go/1.15.3/libexec/src/os/signal/signal_unix.go:49
os/signal.Notify.func1(0x2)
/usr/local/Cellar/go/1.15.3/libexec/src/os/signal/signal.go:144 +0x88 fp=0xc26a5038c0 sp=0xc26a5038a0 pc=0x5615888
os/signal.Notify(0xc1bdd2cd20, 0xc26a503ad8, 0x2, 0x2)
/usr/local/Cellar/go/1.15.3/libexec/src/os/signal/signal.go:164 +0x162 fp=0xc26a503938 sp=0xc26a5038c0 pc=0x56152a2
github.com/gohugoio/hugo/commands.(*commandeer).serve(0xc000adae00, 0xc000c00fc0, 0x2a9, 0x400)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/server.go:498 +0x625 fp=0xc26a503b58 sp=0xc26a503938 pc=0x564e5a5
github.com/gohugoio/hugo/commands.(*serverCmd).server(0xc000c00fc0, 0xc0002c9600, 0xc000597260, 0x0, 0x2, 0x0, 0x0)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/server.go:274 +0x2b6 fp=0xc26a503ca0 sp=0xc26a503b58 pc=0x564cd76
github.com/gohugoio/hugo/commands.(*serverCmd).server-fm(0xc0002c9600, 0xc000597260, 0x0, 0x2, 0x0, 0x0)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/server.go:131 +0x52 fp=0xc26a503ce8 sp=0xc26a503ca0 pc=0x565d472
github.com/spf13/cobra.(*Command).execute(0xc0002c9600, 0xc000597220, 0x2, 0x2, 0xc0002c9600, 0xc000597220)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850 +0x47c fp=0xc26a503dc0 sp=0xc26a503ce8 pc=0x41e0bbc
github.com/spf13/cobra.(*Command).ExecuteC(0xc0003718c0, 0xc000c88c20, 0x8, 0xc0004bfa00)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958 +0x375 fp=0xc26a503e98 sp=0xc26a503dc0 pc=0x41e1735
github.com/gohugoio/hugo/commands.Execute(0xc00019c010, 0x3, 0x3, 0x400b985, 0xc0000b0058, 0x0, 0x0)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/hugo.go:90 +0xb9 fp=0xc26a503f28 sp=0xc26a503e98 pc=0x56397f9
main.main()
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/main.go:23 +0x76 fp=0xc26a503f88 sp=0xc26a503f28 pc=0x565ebb6
runtime.main()
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/proc.go:204 +0x209 fp=0xc26a503fe0 sp=0xc26a503f88 pc=0x403e9e9
runtime.goexit()
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc26a503fe8 sp=0xc26a503fe0 pc=0x4074921
goroutine 40 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc0003965a0)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/go.opencensus.io@v0.22.0/stats/view/worker.go:154 +0x105
created by go.opencensus.io/stats/view.init.0
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/go.opencensus.io@v0.22.0/stats/view/worker.go:32 +0x57
goroutine 56512 [select]:
github.com/gohugoio/hugo/watcher.(*Batcher).run(0xc1d29b1460)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/watcher/batcher.go:53 +0x15f
created by github.com/gohugoio/hugo/watcher.New
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/watcher/batcher.go:42 +0x125
goroutine 56511 [syscall]:
syscall.syscall6(0x423c7c0, 0xb, 0x0, 0x0, 0xc207b6ee88, 0xa, 0x6e41850, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.15.3/libexec/src/runtime/sys_darwin.go:85 +0x2e
golang.org/x/sys/unix.kevent(0xb, 0x0, 0x0, 0xc207b6ee88, 0xa, 0x6e41850, 0xffffffffffffffff, 0x61b3c40, 0x6d86ac0)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/golang.org/x/sys@v0.0.0-20200501145240-bc7a7d42d5c3/unix/zsyscall_darwin_amd64.go:292 +0xa6
golang.org/x/sys/unix.Kevent(0xb, 0x0, 0x0, 0x0, 0xc207b6ee88, 0xa, 0xa, 0x6e41850, 0x0, 0x61b3c00, ...)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/golang.org/x/sys@v0.0.0-20200501145240-bc7a7d42d5c3/unix/syscall_bsd.go:413 +0x71
github.com/fsnotify/fsnotify.read(0xb, 0xc207b6ee88, 0xa, 0xa, 0x6e41850, 0x0, 0x0, 0x0, 0x61b3c40, 0x6d86ac0)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/fsnotify/fsnotify@v1.4.9/kqueue.go:511 +0x6e
github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc1d2a191a0)
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/fsnotify/fsnotify@v1.4.9/kqueue.go:274 +0x831
created by github.com/fsnotify/fsnotify.NewWatcher
/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/fsnotify/fsnotify@v1.4.9/kqueue.go:62 +0x199
goroutine 56513 [select]:
github.com/gohugoio/hugo/commands.(*commandeer).newWatcher.func1(0xc1d29b1460, 0xc000adae00, 0xc1cd9ac618, 0xc1d4f85050)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/hugo.go:873 +0xd9
created by github.com/gohugoio/hugo/commands.(*commandeer).newWatcher
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/commands/hugo.go:871 +0x2cc
goroutine 56858 [select]:
github.com/gohugoio/hugo/livereload.(*hub).run(0x6e02660)
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/livereload/hub.go:39 +0x1e9
created by github.com/gohugoio/hugo/livereload.Initialize
/private/tmp/hugo-20201105-31737-102qobj/hugo-0.78.1/src/github.com/gohugoio/hugo/livereload/livereload.go:98 +0x45Which isn’t incredibly helpful and harder to debug than the last error. Given I’m running the Big Sur beta I figure it’s possible there may be some kind of corner case there so I decided not to open an issue against Hugo. Leaving me at a loss for how to proceed. So I tried some percussive maintenance by increasing max limits at boot by tinkering with launchctl and using recommended defaults:
sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist && \
sudo launchctl load -w /Library/LaunchDaemons/limit.maxproc.plist
And that finally fixed the issue:
| EN
-------------------+--------
Pages | 57227
Paginator pages | 0
Non-page files | 57289
Static files | 37
Processed images | 0
Aliases | 0
Sitemaps | 1
Cleaned | 0
Built in 147644 ms
Watching for changes in /Volumes/Inetpub/textfiles.bit/{archetypes,assets,content,data,la
youts,static,themes}
Watching for config changes in /Volumes/Inetpub/textfiles.bit/config/_default
Environment: "development"
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:64992/ (bind address 127.0.0.1)
Press Ctrl+C to stop
Giving me the ability to update 57,000+ pages I hadn’t touched since I created them on Invisible Arch Linux where things JustWorked™️, reminding me once again just how finicky MacBooks can be and why I prefer to develop software on Linux.
