intro
this document, and most likely, series of posts derived from it will
chronicle the trials and tribulations of getting this research project
both back up and running, and also adding new features into it.
so what is going on here?
i was commissioned to do a bit of a science project... the project,
which for anonymity i will call 'vt' is, in short, is to compare the
performance of a selection of scripting languages in a series of
standardized tests in the context of them being embedded in C++.
progress
- make it go
- angelscript
- boot test
- load test
- load
- compile
- nbody
- vector3 class
- write it in angelscript
- find out days later that wont work and move it to
C++
- planet class
- write it in angelscript
- find out days later that wont work and move it to
C++
- make an array of planets to represent the solar system
- do that
- "register"
- win the battle against cmake preventing the C++
code from compiling when using the register
functions - call the register functions
- win the battle against cmake preventing the C++
- compile without errors?
- get the compiler to not suck
- find out why it isnt compiling and make it not do
that- figure out why it isnt compiling
- make it not do that
- cheat and rewrite the entire script in
imperative style instead.
- cheat and rewrite the entire script in
- vector3 class
- luau
- find information on how to get it going
- find any information at all
- read difference list on how it differs from lua 5.1
- read lua 5.1 docs to understand its system.
- read differences
- read difference list on how it differs from lua 5.1
- find information that tells me something useful
- find any information at all
- find out why it wont compile on its own.
- find information on how to get it going
- haxe (optional)
- quickjs
- v8, spidermonkey, hermes
week 1
day 1
day was spent largely dealing with external factors outside of my
control which turned into a typical boondoggle that ate the entire day
what time was available was spent with the following: continuing reading
the google benchmark documentation, cloned repo to local system.
thoughts on google's benchmark library
- i have to note that for whatever reason, the google documentation
(user guide) seems to be written with the assumption you are already
familiar with their system. (why wouold you even need the intro then
if it isnt going to actually introduce you to anything?) - i am yet to find any explanation to what is contained within their
benchmark::State type, or what their Args() function even does. - i have gathered, that in order to get the metrics we need (both time
and memory), i have no choice but to use json (ugh), and on top of
that, their csv output mode is deprecated! - from the code snippets, apparently, whatever the state reference is,
you can loop over it. - all said however, it appears that the benchmark library/system has
several rather nifty features.
there are some notes about multithreading in here that i should
remember. they may become relevant during the luau tests when it comes
time to do that.
the benchmark system makes references to "installing" it.. is it meant
to be a standalone application? or is it meant to be slapped into the
codebase. im assuming the latter based on how vt is arranged but the
documentation is really unclear on this.
https://github.com/google/benchmark#installation
thoughts on cmake's documentation
it sucks
other thoughts
- on top of all that, il need to see how im expected to actually run
all this with cmake. il probably need to do a dive into cmake docs
as well. - on a related note, it seems that vt is intended to be compiled with
clang. i wonder if this will cause issues as the only compiler i
have on my system where i intend to do this work has gcc.
stuff i did today, in order.
-
i put the repo on my linux machine, its a desktop so im not always
at it today. -
i followed along with the above installation guide, and then
transferred the entire repo into the deps folder to make up for the
missing dependency. i serously hope that is what im actually
supposed to do. -
still no dice on cmake. i tried following along with cmake's
manual
to no avail yet.- trying actually reading instead of skimming and making a second
project on my laptop while im out.
- trying actually reading instead of skimming and making a second
-
AHA! ive gotten 'cmake .' to complete without errors finally. it
took:- cloning benchmark directly into deps, then compiling it as per
google's instructions. (benchmark warns that my version of gcc
is too new for it, assembly tests may not work) - cloning wren directly into deps because it was missing too
apparently
- cloning benchmark directly into deps, then compiling it as per
-
still no clue how to actually run the thing.. time for more reading.
-
ok so to run the thing: cmake --build . -t TARGET NAME HERE for
example, to test lua 5.1: cmake --build . -t test-lua51 the
target names are listed in CMakeLists.txt as the first arg to
the add_executable_() command. this produces an exec of the name
of the build target in the root dir.the test actually ran! kinda... it errored out being unable to
find nbody.lua for some reason. maybe i need to move the exec
into the same folder as nbody.lua? yep, thats exactly what i
have to do. we even got some nice preliminary resuts!the output warns about CPU scaling being enabled. theres
probably an arg in there somewhere to fix that i hope, and the
library being built in debug mode (i explicitly did not do that,
maybe i need to compile the main program explicitly as release?)
(i copied down this table by hand as im writing from another
computer, also it output to console anyway which is something im
going to have to change)
benchmark time CPU iterations BM_CreateVM 104679 ns 104535 ns 6310 BM_LoadNBody 241197 ns 240531 ns 2880 BM_RunNBody/1 0.040 ms 0.040 ms 17620 BM_RunNBody/10 0.139 ms 0.139 ms 4995 BM_RunNBody/100 1.13 ms 1.13 ms 617 not bad for an i7-2600s and a mechanical hard drive! though the benchmark misreports my CPU as running at 3800 MHz when its actually clocked at 2800 MHz. (boost clock?) i went on to the github page for vt and saw there were already instructions on what to do to get it going. i tried it and it didnt really do anything. in any case, im going to tick that one off of the todo list. its getting late so il start on angelscript tomorrow. -
day 2
cmake again :)
ok ive learned some more about cmake, mostly through trial and error.
but in the process ive got angelscript's first bench to compile and
run! process for adding a new language: add a bunch of shit to
cmakelists.txt
near the top of the file, add:
add_subdirectory(deps/languagename/path_to_cmakelists.txt's parent folder)
at the end of the file, add:
add_executable(test-languagename
languagename/benchmarks.cpp
)
target_link_libraries(test-languagename languagename benchmark_main)
add_test(NAME languagename
COMMAND test-languagename ${TEST_ARGS} --benchmark_out${CMAKE_BINARY_DIR}/languagename.json
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/languagename"
)
i found out how to force output to be in json format. add
--benchmark_format=json to the command when running the benchmark. i
get the feeling there is an intended way to execute all this via one
command from looking at cmakelists.txt however im yet to see how to
invoke it...
angelscript
ok ive done some cursory reading on whatever the hell angelscript is,
and its got some of the most galaxy brained design decisions ive ever
seen. no strings??? no dynamic arrays?? you gotta make those yourself
apparently. im reallly not sure who angelscript is designed for. its
simple enough to get going though... the documentation didnt actually
say how to go about including the angelscript library into your project
other than to use static linking. im going to just dump all of
angelscript's code into a subfolder i made for it in deps and hope for
the best. the sdk folder does include a cmake config but im not really
sure what im supposed to do with it. the documentation doesnt say. im
leaving it alone for now
cmake seems to be cromulent with finding angelscript, i did have to
point it in the direction of that cmake subfolder, which it seems to be
happy with for now. my LSP continues to freak out about being unable to
find any of the classes relating to angelscript in the test code. (if
the lsp cant find it, the compiler probably wont either) im going to
need to find a way to get past this issue with angelscript. not having
arrays is a complete nonstarter for the n-body test. angelscript can
apparently register arrays of "handles" to objects. does this mean any
class of note has to be a c++ class?
day 3
angelscript again
today i continue trying to wrap my head around this terrible language. i
will get this damn nbody system going if it kills me! ok, i have learned
that angelscript is actually quite modular, and does come with several
premade add-ons inside it. reegistering is actually quite simple, if
completely undocumented as to how to do it. just include the header for
the feature you want to register, look through the header for the
function to register it, then call it. im yet to figure out how to get
more than a simple error code from the angelscript compiler. wowie, -1!
how informative.
on another note, i "completed" translation of the nbody sim from lua
to angelscript. by completed, i of course mean finished typing only the
essential functions, and have no clue why it wont compile because the
angelscript compiler just returns -1. all of that is moot anyway because
now that im actually registering array and math support, cmake decided
it was no longer happy. who do i have to sacrifice to the linker gods?
yet more cmake learnings
you can add directories with target_includedirectories_(execname-here
PRIVATE PATH-1 PATH-2 PATH-n ) where path is relative to the project
root. it is worth noting that this does [not]{.underline} scan
subdirectories! my LSP seems to complain about not being able to find
the second path, but it compiled fine so whatever i guess.. my LSP
calmed down, for some unknown reason, the include is working, cmake
errors out about their being undefined references to functions in the
headers i included... i swear, linking is the worst part of dealing with
c++ by far.
i end the night well past midnight having not found a solution to the
linker issue. 'undefined reference to'... words to ponder...
git
i issued a commit finally, i might have accidentally included every
dependenccy instead of just benchmark and angelscript as tracked files,
which i get the feeling will screw up whatever weird magic-linking is
going onn on the actual github repo.
why does git commit use a really scuffed version of vi anyway?
day 4
cmake battle
yesterday ended with a weird issue with cmake, and ive never seen stack
overflow be more useless. i wish the cmake docs were written in a sane
fashion. victory in battle! i had to execute a file command to manually
say hey, grab all the files in this directory. i also had to turn off
all the compiler flags because angelscript's library code is poorly
written, or otherwise incompatible with them.
angelscript
angelscript requires a lot of babying. its just hoop after hoop after
hoop after hoop to jump through just to get two (2) classes to exist,
and use one of them in a very not dynamic array. why in the name of
sanity would i ever need to overload operator=? the default behaviour
should be fine, but its not good enough for angelscript apparently.
since im at it, i might replace my math methods with operator overloads.
many more hours spent and all thats been dealt with. but for some reason
the code still fails to run. "free() invalid pointer" thats it, the
entire error message. not even a damn line number. hey, looks like
angelscript comes with a debugger! its only set up to be compiled with
freakin visual studio though, go figure. the verdict is in, angelscript
is complete dogshit, i would only reccommend it to my worst enemies. its
performance characteristics are completely irrelevant because the
language cannot feasibly be used for trivial or nontrivial tasks. there
are no print statements, only a very janky debugger there are no arrays
possible with classes declared within AS. there are no ways to compare
an instance of a c++ class to null. there are no ways to do anything of
note outside of being a massochist.
OS
my system does not seem to be outputting core dumps, zsh says it does
every time the AS program crashes, but its nowhere to be found. ulimit
is infinite. systemd has default config for dumps. coredumpctl says they
are there. but where is there? turns out my core dump system was working
fine all along. core dumps are generated in
"/var/lib/systemd/coredump" they are compressed with zstd, using
"zstd -d filename" decompresses them. all that song and dance just to
find that gdb has no fuckin clue what caused the crash, other than that
it ended up in /usr/lib/libc.so.6 either that or i need to deep dive
into the user manual for gdb.
day 5
i was busy this day. Free 🇵🇸
day 6
gdb
ive concluded based on the experience of the previous day that i need to
study gdb more closely to determine what is wrong here. so gdb has a
handy little tui mode with -tui command line parameter from this i was
able to see that the code reached the destructor for planet and
segfaulted. i removed the offending code and now it just vaiguely
segfaults somewhere within the interpretation process. il need to see
how to set breakpoints in gdb. there is also a full gui frontend called
ddd, il give that a whirl. wow, it looks very 1990s. seems to work
though.
the debugger has told me that AS tried to free something and
segfaulted instead. i do not know what, but i was at least able to
verify that my own destructors seem to terminate properly.
poking around shows me that the script engine tried to do an unaligned
free on a void star 'obj' ddd does not tell me what userFree() is,
instead the next level down the call stack is free() from libc.so.6 so
instead il go up the stack. next up the stack shows that this is the
second to last line in an absolute monster of a function called
CallSystemFunction() fortunately this function has a big ol block
comment above it explaining what the parameters and expected return
values are. unfortunately, i cant quite make heads or tails of anything
actuallg going on in here. i feel that i would need to completely
reverse engineer angelscript in order to determine what is going on
here. angelscript seems to really like sprinkling void * variables
everywhere, making it nigh impossible to go back to see what they are
actually supposed to represent.
angelscript
for now, as it seems like a fruitless endeavour, having debugged for an
hour amd a half and gotten nowhere, i am going to just toss the entire
nbody script into the bin except for use in the earlier tests where it
works fine for the load tests. instead im going to code the test again,
flat, as is done in the lua test.
ok so after doing that, it ran almost first try, sans one pesky syntax
error.
and, the results are in, it runs like shit!
| Benchmark | Time | CPU | Iterations |
|---|---|---|---|
| BM_CreateVM | 127890 ns | 127270 ns | 5344 |
| BM_LoadNBody | 4104441 ns | 4091336 ns | 177 |
| BM_RunNBody | 3470528 ns | 3464377 ns | 202 |
for reference, the final RunNBody, is equivalent to the lua /100 test.
as such, in an apples to apples comparison, we can see that the lua code
is at least thrice as fast! (3.47ms vs 1.13ms) finally, a scientific
result!
and with that, i can finally put this garbage language behind me.. lets
see what is next on the list.
luau
onwards to luau land! i already went ahead and cloned it, copying over
the config from the other lua versions to see if it is just as easy as
that. no documentation exists whatsoever on embedding luau into a
program. according to a github comment on an issue posted in may, there
are several differences between luau and lua from the C++ perspective.
the commentter said they created a program using luau, and lo and behold
it is online. il download it tand see if i cant use it as a base to get
luau going. curiously, everything im reading in the code there seems to
be very similar to what is already happening c++ wise, maybe im just not
finding what the correct include path is for cmake to be happy. im going
to try compiling luau, perhaps it needs to be built.
looks like this luau saga is going to be a long one, you work with
something for hald your life and you dont even know half of it
apparently.
aand the build failed!
i may have to nuke this copy and see about finding a stable version
instead of just direclty cloning from main
i finally found some "documentation" issue #251: "Documentation: C
API". a reference to the holy scripture (lua 5.1 reference manual) and
a difference list.
day 7
luau
i started off the day reading some of the lua documentation, and
transferring this file to the same machine i am doing the development
on. so far i havent really gathered much information, im not entirely
sure where to go from here. i tried compiling a release copy of luau and
that also failed to succeed. ah, it cant compile because i cant read.
cool. the readme.md straight up says what to do. AH FUCK I DOUBLY CANT
READ! it even says how to integrate it into a cmake project. what the
fuck, was this changed yesterday or something? i cannot believe i was
this dumb. it is quite
terse in what it says though, still plenty of room for guesswork it
seems. i should probably fix lunar vim at some point too. it got borked
up over time but i dont want to slow down develoopment to do that right
now...
alright, step one, change the includes on the file, done. step two,
figure out what the hell cmake needs to get this going.
cmake fun
ah, luau's AST is giving me the most fun kind of trouble... welcome
back to linker hell! somehow, nothing can find Ast.h... Ast.cpp cant
find Ast.h, despite being in an adjacent folder. im trying out
GLOB_RECURSE this time, maybe it doesnt like that?
after spending far too long making dinner, and then reading luau's
cmakelist.txt, ive figured it out! all i had to do was include Luau.Ast
Luau.Compiler Luau.VM as libraries and it seemed to find them just fine,
no futzing around with GLOBs at all. now, i have a gigantic mess of C++
errors to deal with thanks to the differences between lua and luau
luau again
oh what in tarnation... why is the compiler reading from /usr/lib??? why
is my actual lua install involved in this in any way, shape, or form?
and more importantly, how do i make it not do that? ive tried a couple
things, no dice. it isnt CMAKE_IGNORE_PATH or CMAKE_SYSTEM_IGNORE_PATH...
no clue what it could be...
unfortunately, i cannot simply uninstall lua as that breaks other
dependencies on my sytstem. well fuck.
what even happens now? there is literally 0 information i can find on
this one... is it an environment variable thing? can i even set
environment variables for cmake without fucking up my system? are they
even separate from the sysem? if so i might just nuke PATH. ah well i
can find 0 information on this anyway...
maybe someone on
cohost knows?
well, that's all for this week, stay tuned for more! --🦊🔮
