Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - a better makefile for your devkitSMS projects

Reply to topic
Author Message
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
a better makefile for your devkitSMS projects
Post Posted: Fri May 14, 2021 3:30 pm
Last edited by sverx on Wed May 19, 2021 11:13 am; edited 2 times in total
I admit I never really used makefiles for compiling code that uses devkitSMS, but when a project grows over a certain size it starts to be annoyingly long to recompile everything over and over again just because of a small change somewhere which really doesn't require to recompile *everything*.

So I looked into make and makefiles.

Unfortunately, the serious problem with devkitSMS and makefiles is that the assets2banks tool creates the assets2banks.h file, which is a file you'll easily have to #include in many of your sources.

So I wrote an example makefile that has one additional advantage apart from all the basic stuff (will compile only the sources that are indeed modified).

With this, IF you do any changes to the assets that DON'T change the CONTENTS of assets2banks.h, your C sources referencing that file won't be uselessly compiled (this saves a lot of time!) but you'll still get the changed assets linked and the ROM updated (found the right suggestion for this trick here - basically we check the assets2banks.h MD5 hash instead of its timestamp to see if a recompilation of program source files is needed).

MAKESMS=~/SMS/tools/makesms
ASSETS2BANKS=python3 ~/SMS/tools/assets2banks.py
SDCC=~/SMS/sdcc/bin/sdcc
CRT0=~/SMS/SMSlib/crt0/crt0b_sms.rel
SMSLIB=~/SMS/SMSlib/SMSlib.lib
PEEPFILE=~/SMS/SMSlib/peep-rules.txt

to-md5 = $(patsubst %,%.md5,$1)

output.sms: output.ihx
   $(MAKESMS) output.ihx output.sms
   
output.ihx: main.rel banked_code1.rel banked_code2.rel banked_code3.rel bank4.rel bank5.rel bank6.rel bank7.rel
   $(SDCC) -o output.ihx -mz80 --no-std-crt0 --data-loc 0xC000 \
   -Wl-b_BANK1=0x14000 -Wl-b_BANK2=0x24000 -Wl-b_BANK3=0x34000 \
   -Wl-b_BANK4=0x48000 -Wl-b_BANK5=0x58000 -Wl-b_BANK6=0x68000 -Wl-b_BANK7=0x78000 \
   $(CRT0) main.rel $(SMSLIB) \
   banked_code1.rel banked_code2.rel banked_code3.rel \
   bank4.rel bank5.rel bank6.rel bank7.rel
   
main.rel: main.c
main.rel: common.h main.h banked_code1.h banked_code2.h banked_code3.h
main.rel: $(call to-md5,assets2banks.h)
   $(SDCC) -c -mz80 --peep-file $(PEEPFILE) main.c

banked_code1.rel: banked_code1.c
banked_code1.rel: common.h main.h banked_code1.h banked_code2.h banked_code3.h
banked_code1.rel: $(call to-md5,assets2banks.h)
   $(SDCC) -c -mz80 --peep-file $(PEEPFILE) --codeseg BANK1 banked_code1.c

banked_code2.rel: banked_code2.c
banked_code2.rel: common.h main.h banked_code1.h banked_code2.h banked_code3.h
banked_code2.rel: $(call to-md5,assets2banks.h)
   $(SDCC) -c -mz80 --peep-file $(PEEPFILE) --codeseg BANK2 banked_code2.c
   
banked_code3.rel: banked_code3.c
banked_code3.rel: common.h main.h banked_code1.h banked_code2.h banked_code3.h
banked_code3.rel: $(call to-md5,assets2banks.h)
   $(SDCC) -c -mz80 --peep-file $(PEEPFILE) --codeseg BANK3 banked_code3.c

assets2banks.h.md5: assets2banks.h
   @$(if $(filter-out $(shell cat $@ 2>/dev/null),$(shell md5sum $<)),md5sum $< > $@)

assets2banks.h bank4.rel bank5.rel bank6.rel bank7.rel: ./assets/*
   $(ASSETS2BANKS) assets --firstbank=4 --compile --singleheader


Feedback/suggestions welcome! (also, I'm a makefile beginner so if this file can be improved, I'm all ears! :D )
  View user's profile Send private message Visit poster's website
  • Joined: 24 Mar 2021
  • Posts: 120
Reply with quote
Post Posted: Fri May 14, 2021 7:24 pm
In my opinion, the thing that makes Makefiles really nice is the ability to write generic rules using "automatic variables". For a simple SMS project, I wrote this:

%.sms: %.ihx
        $(IHX2SMS) $< $@

%.asm %.sym %.lst %.rel: %.c
        sdcc -c -mz80 --std-sdcc99 $<

%.rel: %.s
        sdasz80 -g -o $<

# plus a bunch of other specific rules that I've not included here

This allows me to say "How can you create an .sms file? If you have an .ihx file ("%.ihx"), run this program on it"
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 14688
  • Location: London
Reply with quote
Post Posted: Sat May 15, 2021 7:02 am
You could make assets2banks implement the check itself - if the generated .h is identical to the existing file, do not write it. In C it’s easy to pretend you’re super concerned about resources so you generate straight into the file pointer, but you can actually do it all in memory easily enough, the file is small, making it easier to compare.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Sat May 15, 2021 11:43 am
lidnariq wrote
In my opinion, the thing that makes Makefiles really nice is the ability to write generic rules using "automatic variables".


That's true, and surely my example makefile could use a bit of that too, but the point was mainly to avoid recompilation of what hasn't changed (and, in this regard, generic rules won't disturb at all!) *and* to avoid recompilation when assets2banks.h gets rewritten with same contents (you won't be able to handle that without some specific workaround).

Maxim wrote
You could make assets2banks implement the check itself - if the generated .h is identical to the existing file, do not write it.


I thought about this too, and I choose to go that different route for many reasons:
- assets2banks it's written in Python and I don't really know Python very well
- I would prefer a simpler approach instead of generating the output and compare it to an existing file
- if I could workaround the problem with no changes to assets2banks code I would do that
- I'm lazy ;)
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Mon May 17, 2021 1:35 pm
question about this line:

lidnariq wrote
%.rel: %.c
        sdcc -c -mz80 --std-sdcc99 $<


how would it understand if a .rel needs to be recompiled on a .h change?
  View user's profile Send private message Visit poster's website
  • Joined: 24 Mar 2021
  • Posts: 120
Reply with quote
Post Posted: Mon May 17, 2021 6:22 pm
sverx wrote
how would it understand if a .rel needs to be recompiled on a .h change [if the .h isn't explicitly mentioned in the prerequisites]?
You're right, it doesn't.

You can separate the "dependencies" rules from the "how to build" rules, if that helps:


%.rel: %.c
  sdcc $<

[...]
specific.rel: specific.c shared.h


Or if you know that all of the c files in any given directory do require a specific header file, you can also

%.rel: %.c shared.h
 sdcc $<
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Tue May 18, 2021 2:48 pm
It makes sense, but at this point I wonder how you tell that some rel files are to be built by compiling C sources and other are to be built by assets2banks utility?
  View user's profile Send private message Visit poster's website
  • Joined: 24 Mar 2021
  • Posts: 120
Reply with quote
Post Posted: Tue May 18, 2021 8:54 pm
You can have multiple rules specifying how to make a output, such as:


%.rel: %.c
 sdcc blah

%.rel: assets2banks.cfg %.bin
  python assets2banks.py blah

Make will only use the ones whose prerequisites exist (or it knows how to make)

If you have a more complicated set of rules, such as
a: b
   cp $< $@

a: c
   cp $< $@

b: d
   cp $< $@

c: e
   cp $< $@

d:
   touch $@

e:
   touch $@
it ends up doing a breadth-first search, ultimately invoking every command except "a from "b"
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Wed May 19, 2021 11:23 am
lidnariq wrote
You can have multiple rules specifying how to make a output, such as:


%.rel: %.c
 sdcc blah

%.rel: assets2banks.cfg %.bin
  python assets2banks.py blah

Make will only use the ones whose prerequisites exist (or it knows how to make)


oh, this is interesting, though a bit confusing at first.
Does this mean that make actually doesn't know which recipe is the correct one to create, say, bank4.rel or banked_code2.rel but as long as there are source files or assets files that are newer than any .rel file it will trigger either one or the other recipe?
  View user's profile Send private message Visit poster's website
  • Joined: 24 Mar 2021
  • Posts: 120
Reply with quote
Post Posted: Wed May 19, 2021 7:59 pm
Make determines which recipe is the correct one based on which recipes can be satisfied. "%" doesn't mean *, but instead means "the exact same text on both sides of the :"

so if you wanted to make bank4.rel, and bank4.bin exists, it could use assets2banks; but if you also had bank4.c it'd pick one of the two.

For example, this makefile:

%.1: %.2
   cp $< $@

%.1: %.3
   cp $< $@

%.2:
   touch $@

%.3:
   touch $@


invoked with "make a.1", will generate a.2, generate a.1, and then delete a.2. If only one of a.2 or a.3 already exist, it'll use that one in preference (and then not delete it).

If a.2 exists and is older than a.1, even if a.3 is newer than a.1, make will think it's done.
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Fri May 21, 2021 10:48 am
lidnariq wrote
so if you wanted to make bank4.rel, and bank4.bin exists, it could use assets2banks


mmm, that's not how assets2banks works, output file names don't come from input file names

but I think if there are two recipes for %.rel it will anyway execute those with some newer file in the dependencies, right?

I mean as in

%.rel: ./assets/*
  assets2banks assets blah blah

%.rel: %.c
  sdcc -c blah blah


if some file in the assets folder has changed, make will launch assets2banks program and if some c sources changed it will launch SDCC... and at the end hopefully all the needed rel files will be created/updated, or am I assuming wrong?
  View user's profile Send private message Visit poster's website
  • Joined: 24 Mar 2021
  • Posts: 120
Reply with quote
Post Posted: Fri May 21, 2021 7:40 pm
sverx wrote
but I think if there are two recipes for %.rel it will anyway execute those with some newer file in the dependencies, right?
I don't usually use actual wildcards, but I think that's correct.
  View user's profile Send private message
  • Joined: 25 Jul 2007
  • Posts: 716
  • Location: Melbourne, Australia
Reply with quote
Post Posted: Sun May 23, 2021 1:03 pm
lidnariq wrote
sverx wrote
how would it understand if a .rel needs to be recompiled on a .h change [if the .h isn't explicitly mentioned in the prerequisites]?
You're right, it doesn't.

You can separate the "dependencies" rules from the "how to build" rules, if that helps:


You can have SDCC generate the prerequisites automatically with the -MMD preprocessor option (it's based on GCC so many of the same flags should work). It will generate a .d file for every source file it compiles which can then be included in the makefile.

This is an example from my current makefile

# program executable file name.
PROG = worms
# compiler
CC = sdcc
# linker
LD = sdcc
# delete command
RM = del /f
# Compiler flags go here.
CFLAGS = -MMD -c -mz80 --peep-file peep-rules.txt
# Linker flags go here.
LDFLAGS = -mz80 --no-std-crt0 --data-loc 0xC000
CRT0 = crt0_sms.rel
SMSLIB = SMSlib.lib
OBJS = $(PROG).rel bank2.rel
DEPS = $(OBJS:.rel=.d)

$(PROG).sms: $(PROG).ihx
   ihx2sms $< $@

$(PROG).ihx: $(OBJS)
   -$(LD) -o $@ $(LDFLAGS) $(CRT0) $^ $(SMSLIB)

%.rel: %.c
   $(CC) $(CFLAGS) $<

.PHONY: clean
clean:
   -$(RM) *.sms *.ihx $(OBJS) $(DEPS)

-include $(DEPS)


This way any changes to the headers will trigger a rebuild.
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 3763
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Thu Sep 23, 2021 9:42 am
so now I discovered the joys of the parallel make (--jobs n) and I can run 4 SDCC instances to build the 4 object files at the same moment using n=4 ... which is very nice since I have a quad core processor.

unfortunately it also invokes assets2banks 4 times to generate the 4 rel files from the assets folders :(

how can I tell make that one rule creates ALL the specified targets in a single invocation? I tried using the &: separator as I read that it's for target groups but I get the same result: 4 invocations of assets2banks tool :(

this is the recipe at the moment:
assets2banks.h bank4.rel bank5.rel bank6.rel bank7.rel &: ./assets/*
   $(ASSETS2BANKS) assets --firstbank=4 --compile --singleheader


Any help from some make guru? Thanks!

edit: oh, wait - it's a feature of GNU Make 4.3, and I'm on 4.2.1 of course...

edit2: once I got Make 4.3 installed and running, the problem was solved.
"Grouped Targets" was indeed (see here).
Basically the rule now says that ALL the listed targets are generated by a single execution of the recipe, so parallelism won't get in the way here. :)
  View user's profile Send private message Visit poster's website
Reply to topic



Back to the top of this page

Back to SMS Power!