Macros

1
Macros

• You might recall from earlier that some of the
  statements we use are not actually functions but
  macros
• In CL, a function evaluates all of its parameters
  and applies them to the function
• if is a macro, not a function
• only one of the two clauses gets evaluated based
  on the condition
• let is a macro
• the variables are not evaluated, only allocated
  and bound
• the initialization code is evaluated but not the
  variables
• dolist and dotimes are not macros
• the (var lis/value) is not a function call
• etc
• Macros are way of writing code that can be used
  to generate code rather than be invoked by code
• Macros are essential in order to grow the
  language
• many programming languages have macro facilities,
  but few languages use them as extensively as in
  Lisp

2
What is a Macro

• A macro is a piece of code, much like a function
• but in the case of a macro, the code is not
  evaluated when entered into the REPL environment
• instead, it is evaluated on demand
• The macro returns a form (code) rather than a
  value
• so that the code returned can be evaluated by
  someone else at a later time
• The macro can be defined so that it builds code
• you can also pass it code to be used as it builds
  code
• for instance, you might want to build a function
  by appending pieces of code to a list and then
  returning the list when done
• So the macro allows us to postpone evaluation
• defining a macro is somewhat like writing a
  function, you use defmacro, include params and
  follow it by the code to be executed

3
Controlling Evaluation

• The is normally used to control evaluation
• We have used to say take literally, do not
  evaluate
• In a macro, we can continue to use the but
  there is a superior operator to use (back
  quote)
• The quote and back quote both say do not
  evaluate what follows
• but the back quote permits the use of the ,
  (comma) which means unsuppress the back quote
• the , will allow us to send a name to a macro
  definition and use that name in place of the
  parameters name
• Deciding when to use and , will be challenging
• if you try to use , outside of a you will get
  an error
• you do not have to use , in your code but if
  not, then it is the same as if you were using
• Note that we could avoid using and , by using a
  combination of list and as shown on the next
  slide but this would get tedious

4
Back Quote and Comma Examples

• (the sum of 5 and 10 is ( 5 10))
• since this is all in the scope of , none of it
  is evaluated, instead it is returned verbatim
• (the sum of 5 and 10 is ( 5 10))
• we get the same result if we use the normal quote
• if we place a , prior to ( 5 10) we get
• (the sum of 5 and 10 is 15)
• (like ) says do not evaluate, just return
  whereas ,x... says evaluate x right now
• (dolist (a lis) (print a))
• returns (dolist (a lis) (print a))
• assume that lis is the list (a b c d), then
  (dolist (a ,lis) (print a)) returns
• (dolist (a (a b c d)) (print a))
• (,x ,y ,( x y))
• if x 3 and y 4 then this returns (3 4 7)
• Notice that we could achieve the same results by
  placing any of these lists into a (list) function
  call and quoting each item in the list except
  those that have a ,
• (list the sum of 5 and 10 is ( 5 10))
• (list x y ( x y))

5
A More Useful Example

• Consider that we want a function that will
  evaluate an expression and return one of three
  values
• First value if expression is negative
• Second value if expression is zero
• Third value if expression is positive
• This is equivalent to FORTRANs first if
  statement
• if(expression) a, b, c
• We cant use defun because all 3 expressions
  would be evaluated
• We could use an if or cond statement though, but
  using a macro is easier

(defmacro if3 (expr val1 val2 val3) (cond
((lt ,expr 0) ,val1) (( ,expr 0)
,val2) (t ,val3)))
If we call if3 as (if3 (- a b) 0 -) this
becomes (or unfolds into)
(cond ((lt (- a b) 0) ) (( (- a b) 0)
0) (t -)
You can see that (- a b) is substituted for expr,
this is called a macro-substitution
6
Multiple Assignment Example

• Assume that you want to assign multiple variables
  to the same expression
• We can do this by (setf a expr b expr c expr )
  but this is inefficient (we are evaluated expr
  several times)
• I could also do (let ((temp (expr))) (setf a temp
  b temp c temp) ) but this can be tedious
• What I want to do is use code like this (setf a b
  c expr)
• setf however does not work properly, so instead I
  will define a macro

(defmacro assign2 (v1 v2 exp) (let ((temp
,exp)) (setf ,v1 temp) (setf ,v2 temp)
temp)) (defmacro assign-many (exp v1 optional
v2 v3 v4) (let ((temp ,exp)) (setf ,v1
temp) (if (not (null ,v2)) (setf ,v2 temp)) (if
(not (null ,v3)) (setf ,v3 temp)) (if (not (null
,v4)) (setf ,v4 temp)) temp))
Returns the value assigned Up to 4
variables can be assigned, only one is
required Notice the order, exp comes first
7
Variable Assignment Example

• Here I want to choose which of two variables to
  assign to the other
• For instance, if a lt b, then assign a to b
  otherwise assign b to a
• Code (if (lt a b) (setf a b) (setf b a))
• As a macro

(defmacro assigntwoways (v1 v2 test)
(progn (if ,test (setf ,v1 ,v2) (setf ,v2
,v1)) ,v1))
Notice the use of progn here, it is needed
because of the ,v1 at the end without (progn, CL
would think that the was only in front of the
if statement meaning that the ,v1 has a ,
outside of a scope Since this code unfolds
into an if statement which executes a setf, v1 or
v2 will change as opposed to writing a defun
statement where we would have to pass back v1
and/or v2 to be stored again
8
Writing a Macro

• So far it doesnt seem difficult to write a macro
  but
• how do you know when to use and ,
• how do you know what code to use
• how do you know when to use progn or let
• Start by writing the code that you want to have
  produced in a given setting
• (cond ((lt (- x y) 0) x) (( (- x y) 0) y) (t ( x
  y)))
• Now make it into a function with parameters to
  replace the various items (the statements x, y,
  ( x y) and the expression (- x y)) with
  variables
• Since the variables are not to be evaluated by
  the macro, but substituted when the macro is
  called, insert a , before them
• Since we have , in our code, we have to -- find
  a location where will cover the scope of all
  parameters that have a ,
• Replace defun with defmacro

9
Example

• I want to write a do loop that will iterate while
  the loop variable is positive
• (do ((x init step)) ((lt x 0)) code)
• I want this code to be generated whenever anyone
  wants to use it by passing the variable, the init
  value, the step function, and the code

(defun while-positive (x init step code)
(do ((x init step)) ((lt x 0)) code)) The defun
has a flaw, x in the loop is not the same as
the x passed as a parameter, I need macro
substitution (defmacro while-positive (x init
step code) (do ((,x ,init ,step)) ((lt ,x
0)) ,code))
Call this with (while-positive y 10 (- y 2)
(print y))
10
Using Body and ,_at_

• So far, the body of our construct (the executable
  code inside the loop or if statement) has been a
  single statement
• this approach does not work if we have lists of
  executable statements
• for instance, (while-positive x ( x 1) (setf a
  ( a x)) (print a)) does not work because there
  are two items where the macro expects the single
  item code
• to accommodate multiple statements as the body,
  we could use rest (all of the following code is
  placed into a single list of functions)
• instead, we use a variation called body
• the only difference is that body pretty prints
  better than rest
• We supply a single parameter after this which is
  then the collection of all items passed
• inside of the scope, we can apply the body by
  doing ,_at_parameter

11
Examples Using ,_at_ vs
Backquote Syntax Equivalent List-Building Code Result
(a ( 1 2) c) (list 'a '( 1 2) 'c) (a ( 1 2) c)
(a ,( 1 2) c) (list 'a ( 1 2) 'c) (a 3 c)
(a (list 1 2) c) (list 'a '(list 1 2) 'c) (a (list 1 2) c)
(a ,(list 1 2) c) (list 'a (list 1 2) 'c) (a (1 2) c)
(a ,_at_(list 1 2) c) (append (list 'a) (list 1 2) (list 'c)) (a 1 2 c)
12
Code Examples

• A number of the operations we have been using are
  macros by using ,_at_body
• (defmacro when (condition body body) (if
  ,condition (progn ,_at_body)))
• (defmacro unless (condition body body) (if (not
  ,condition) (progn ,_at_body)))
• (defmacro dotimes ((var1 var2) body body) (do
  ((,var1 0 ( ,var1 1))) (( ,var1 ,var2))
  ,_at_body))
• Notice the extra layer of ( ) in the parameters
  to fit the dotimes form (dotimes (a 10) )
• (defmacro dolist ((var1 lis) body body) (do
  ((,var1 (car ,lis) (car ,lis))) ((null ,lis))
  (setf ,lis (cdr ,lis)) ,_at_body))
• the _at_ is used in conjunction with , to say
  evaluate the following as a group
• Many implements of CL allow you to use rest
  instead of body but use body just to play safe,
  it also helps when pretty-printing

13
Destructuring

• Notice in some of the previous examples that we
  wrapped our params in an extra set of ( )
• When binding the formal parameters (those in the
  macro header) to the actual parameters (those in
  the macro call), CL does something called
  destructuring it finds the matching parameters
  based on embeddedness
• (defmacro dotimes ((var1 var2) body body) (do
  ((,var1 0 ( ,var1 1))) (( ,var1 ,var2))
  ,_at_body))
• (dotimes (x 10) ) var1 gets x, var2 gets 10
• Destructuring can be done to any arbitrary
  nestedness
• consider some examples
• (defmacro m1 ((a b) (c d) (e f)) ) called with
  (m1 (1 2) (3 4) (5 6))
• (defmacro m2 (a (b c) ((d) e)) ) called with (m2
  1 (2 3) ((4) 5))
• (defmacro m3 (a (b optional c) (optional d)) )
  called with (m3 1 (2) ( )) or (m3 (1 (2 3) ( ))
  or (m3 (1 (2) (3))

14
More Macro Examples

• Consider that we want to write a counter
  controlled loop that iterates over the sequence
  of prime numbers
• (do ((i 2 ( i 1))) (( i n)) (if (prime i) (b
  i)))
• b is the body of code to execute on each prime
• This becomes the macro
• (defmacro doprimes ((var limit) body b) (do
  ((,var 2 ( ,var 1))) (( ,var ,limit)) (if
  (prime ,var) ,_at_b))
• Now consider the same thing over a sequence
  generated by some function f rather than prime
• (defmacro dosequence ((var limit f) body b) (do
  ((,var 2 ( ,var 1))) (( ,var ,limit)) (if
  (funcall ,f ,var) ,_at_b)))
• could be called by (dosequence (a 100 evenp)
  (print a))
• (defmacro dosequence ((var limit f) body b) (do
  ((,var 2 ( ,var 1))) (( ,var ,limit)) (if
  (funcall ,f ,var) ,_at_b)))
• could be called by (dosequence (a 100 evenp)
  (print a))

15
Building Code From A List

• Lets make a simple switch statement in CL
• In Java/C, we do switch (var) case val1
  statement break case val2 statement break
  
• Here, we want to be able to say (switch x val1
  statement1 val2 statement2 )
• we dont need to add the extra syntax of case
  and ,
• to simplify matters, we will assume that each
  statement is a single operation, and to make it
  even easier, merely a return value (say a number)
  
• we will also simplify the syntax by permitting
  the values and return values to be stated outside
  of a list
• (switch x 1 1 2 20 3 300 4 -1) x 1 returns 1,
  x 2 returns 20, x 3 returns 300, x 4
  returns -1
• Problem we cant just use ,_at_body here because
  we need to divide the body up into separate
  statements that say (( x 1) 1) (( x 2) 20) ((
  x 3) 300) etc
• So rather than relying on ,_at_body, we will build
  our body through a do loop

16
Building Our Macro

• It seems like we could write any function to
  build a cond statement for us
• (let ((temp (cond)))
• (do (setf temp (append temp (( x 1) 1))
• However, we need to be able to insert the
  variable name provided by the caller as well as
  the unique list of values/return values
• So a macro it is!
• In the macro though, we have to differentiate
  whether we are appending to a local variable, or
  manipulating a parameter
• The former is done outside of the scope, the
  latter is done within the scope

Do lis elements in pairs, (nth i lis) being the
value, used in ( var val) and the next element
being the return value
(defmacro switch (var body lis) (let ((temp
'(cond))) (do ((i 0 ( i 2))) ((gt i (length
lis))) (setf temp (append temp ((( ,var
,(nth i lis)) ,(nth ( i 1) lis)))))) temp))
Notice this is the only part of the macro that
has scope
17
An Alternative Approach

• The previous approach was concise but perhaps not
  as easy to understand as it could be
• Here we more clearly build each switch argument
  as two separate lists, one storing ( var item)
  and the other being an executable statement
• We then tack each together into a list and then
  append it to the growing cond statement

When called with (switch x 0 (print a) 1
(print b) 2 (print c)) unfolds into (cond ((
x 0) (print a)) (( x 1) (print b))
(( x 2) (print c)))
(defmacro switch (var body b) (let ((temp
'(cond)) (count 0) temp2 temp3 item) (do ( )
((null b)) (setf item (car b)) (setf b
(cdr b)) (if (evenp count)
(setf temp2 ( ,var ,item)) (progn (setf
temp3 (append (list temp2) (,item)))
(setf temp (append temp (list temp3)))))
(setf count ( 1 count))) temp))
18
Improving Our Macro

• What if we want to permit a default?
• Since the default case should be the last in our
  list, we will exit our loop once we find the
  default
• Notice the use of t this permits a default
  case to be selected when default is found in the
  list
• we could alternatively use default if we want
  the programmer to use the word default when
  calling the macro
• We could invoke this macro with (switch2 x 1 10 2
  20 t 30)
• if x were 2, we would get 20, if x were 5, we
  would get 30

(defmacro switch2 (var body lis) (let ((temp
'(cond))) (do ((i 0 ( i 2))) ((gt i
(length lis))) (if (equal (nth i lis)
t) (setf temp (append temp ((t ,(nth ( i 1)
lis))))) (setf temp (append temp (((
,var ,(nth i lis)) ,(nth ( i 1) lis))))))) temp))
19
Macro Expanding

• When you define a macro, you are returning a new
  form which can be called directly
• From the REPL interface
• From another function
• But the macro is actually performing
  substitution by taking what you defined and
  substituting parameters
• This is done through macro-expansion
• What is returned is then executed
• You can view the intermediate form without
  execution by using macroexpand-1
• (macroexpand-1 (macroname params))
• For example, I want to see the code generated by
  doprimes passing it the parameters (n 12) and
  (print n)
• (macroexpand-1 '(doprimes (n 12) (print n)))
• (DO ((N 2 ( N 1))) (( N 12)) (IF (ISPRIME N)
  (PRINT N)))
• T
• Notice that macroexpand-1 prints out the macro as
  expanded by the parameters, and then returns T
  (or nil if there is a failure)

20
Using Macros to Define Functions

• To this point, our macros have expanded into
  function calls and the return of the function
  call is what the macro call returns
• consider switch which expands to a cond statement
  that is then executed
• We can also use macros to generate defun
  statements
• this lets us generate functions to be called
  later
• Consider a database using a list of structures
• we could define search functions for all of our
  slots, but this could be time consuming
• or we could define a macro that generates various
  search functions for us
• How?
• we need to have access to all of the structures
  slots and their types, for each slot, we generate
  one or more defun statements from our macro

21
Starting The Macro

• Consider that we have a student structure which
  stores the slots name, major, hours, gpa of
  types string, string, integer, number
• We want to define search functions similar to the
  following
• However, we dont want to have to write them all
• So we want a macro which will generate defun
  statements
• This macro will need to know the names of the
  slots
• in this way, we can access the structures slot
  as we did above with (student-hours a)
• And the type of each slot
• so that we know which comparison functions to
  generate (an integer will probably require lt, gt,
  and , but a string might include others such as
  contains, does-not-contain,

(defun search-hoursgt (db target) (dolist (a db)
(if (gt (student-hours db) target) (print
a))))
22
Partial Macro Solution
(defmacro generate-a-search (struct-name
slot-name slot-type) (let ((function-name
(intern (concatenate 'string (symbol-name
struct-name) "-" (symbol-name slot-name)
"gt"))) (compare-type (if (or (equal slot-type
'number) (equal slot-type 'integer)) 'gt
'Stringgt)) (accessor-name (intern (concatenate
'string (symbol-name struct-name) "-"
(symbol-name slot-name))))) (defun
,function-name (db target) (dolist (a db)
(if (apply ,compare-type (,accessor-name a)
target) (print a))))))
Now, we call this for each slot-name/slot-type in
the structure Example (generate-a-search
STUDENT MAJOR STRING) We have to expand on this
if we want to go beyond simple gt, lt, searches
23
Continued

• Lets go beyond the search producing macro
• We would like to pass a structural definition to
  a macro, it can produce for us an entire database
• The struct definition
• Functions for search
• A function to print the structure in a formatted
  way
• A function to test two structures for equality
  (equal if they have the same slot values)
• A function to input a new structure
• Etc
• The structural definition should include the slot
  names, but here we might also want to include
  each slots type
• we can decide to what search functions to produce
  (e.g., a search based on whether a string
  contains a substring rather than simply stringlt,
  stringgt, string)
• we can use the proper form of equality (, equal,
  char, string, etc)
• we can generate a proper format statement for an
  output function
• On the next slide, we have a macro that produces
  the defstruct and an equals function
• Notice the weird use of ,variable
• This became necessary in order to pass parameters
  properly

24
Example Continued
(defmacro generate-struct (name slots types)
(let (slot type temp1 temp2 fname (s1 (gensym))
(s2 (gensym)) andlist eq-func) (setf fname
(intern (concatenate 'string (symbol-name ,name
-equals)))) (setf andlist '(and))
(dotimes (i (length slots)) (setf slot
(nth i slots)) (setf type (nth i
types)) (if (or (equal type 'integer)
(equal type 'number)) (setf eq-func ') (if
(equal type 'character) (setf eq-func 'char)
(setf eq-func 'string))) (setf temp1
(list (intern (concatenate 'string (symbol-name
,name) "-" (symbol-name ,slot))) ,s1))
(setf temp2 (list (intern (concatenate 'string
(symbol-name ,name) "-" (symbol-name ,slot)))
,s2)) (setf andlist (append andlist
(list (,eq-func ,temp1 ,temp2))))) andlist
becomes (and (equal (slot-name1 s1) (slot-name1
s2)) ) for each slot (progn (defstruct
,name ,_at_slots) produce the defstruct
(defun ,fname (,s1 ,s2) ,andlist)))) produce
the equal function
25
A Problem

• Consider the following macro definition
• (defmacro foo (a b) (let ((sum 0)) (do ((,a 1 (
  1 ,a))) (( ,a ,b)) (setf sum ( sum ,a))) sum))
• This macro merely constructs a do loop that
  iterates from 1 to b, summing up the values and
  returning the sum obviously we dont need a
  macro for this, but it illustrates an important
  point
• No big deal right?
• If we were to do (foo a 10) then it sums up 1-9
  and returns 45 as we would expect
• What do you suppose will happen if we do (foo sum
  10)?
• Lets macro expand this (foo sum 10) and find
  out
• (LET ((SUM 0)) (DO ((SUM 1 ( 1 SUM))) (( SUM
  10)) (SETF SUM ( SUM SUM))) SUM)
• Notice our terminating condition is not what we
  had expected instead of stopping once the loop
  variable reaches 10 because our loop variables
  name is replaced with sum
• And the problem here is that ( sum 10) is never
  true (sum takes on the values 1, 3, 7, 15, 31,

26
The Problem Same Named Items

• The problem with our last example was that a
  parameter that we passed to the macro to
  substitute for ,a happened to be the same name as
  a local variable
• What are the odds of that happening?
• Probably not very high, but if it does happen, it
  produces behavior that we dont want
• The solution?
• Renaming the parameter isnt an adequate solution
  because we cant control what the other
  programmers might do
• and since we cant predict what they will do, we
  need to play safe
• Renaming the local variable would work if we
  could come up with a name that we are guaranteed
  that they will never ever use
• how about Slartibartfast??? Ok, someone out
  there might be compelled to name a variable this!

27
The Solution Gensym

• Gensym is a CL function that generates a unique
  symbol, one that is not used elsewhere in the
  system
• These symbols will differ from user defined
  symbols because they have G generated followed
  by a unique number as in G708
• Using gensym several times in a row will generate
  several symbols with consecutive integer values
  (so that the next would be G709)
• This does not seem very usable what good is it
  to generate a seemingly random symbol?
• Its use is this we will name any local variable
  using gensym so that the name of the local
  variable is unique

(let ((sum (gensym))) now for the remainder of
the macro, refer to ,sum instead of sum, sum is
the parameter passed to the macro, ,sum is a
symbol like G708 there is no way that a
programmer can pass G708 it is disallowed,
therefore we are guaranteed a unique name
28
Another Example

• In this example, there doesnt seem to be a
  problem since we are not declaring local
  variables
• (defmacro cube (n) ( ,n ,n ,n))
• This works fine when called with (cube 2) or
  (cube x) where x was set to some value previously
  or even (cube (- x y)) where x and y were both
  previously set
• But what happens if we call cube as (cube (incf
  x))?
• Lets macroexpand this
• (macroexpand-1 (cube (incf x))) ? (cube (incf x)
  (incf x) (incf x))
• Recall that incf is destructive
• the value of x changes every time it is called
• so if we originally set x to 2, then we get (cube
  3 4 5) ? 60!
• To get around this, lets again use gensym
• (defmacro cube (x) (let ((name (gensym))) (setf
  ,name x) ( ,name ,name ,name)))

29
Revised Macro
(defmacro foo (a b) (let ((sum (gensym)))
(progn (setf ,sum 0) (do ((,a 1 ( ,a 1)))
(( ,a ,b)) (setf ,sum ( ,sum
,a))) ,sum)))

• Here we have pulled the let statement out to
  appear prior to the
• This allows us to generate a symbol for sum, and
  then reference that generated symbol so that we
  dont have to take the chance that the programmer
  called foo with sum
• If we were to macro-expand (foo sum 10) now we
  get
• (PROGN (SETF G829 0) (DO ((SUM 1 ( SUM 1)))
  (( SUM 10)) (SETF G829 ( G829 SUM)))
  G829)
• Notice how G829 has been generated to replace
  our local variable so that our terminating
  condition no longer is affected by that variable

30
Do We Need A Macro?

• Consider this macro
• Do we really need to make it into a macro? Could
  we not just use defun?
• Yes, just use defun so under what
  circumstance(s) do we want to use defmacro?
• Both defun and defmacro generate new pieces of
  code but defmacro gives us the ability to control
  when things are evaluated so that we can make use
  of macro substitution
• If at the time I am writing the code, I do not
  know how the code will be used by another
  programmer, I should use defmacro
• Alternatively, if I have a routine that depends
  on another function (like dosequence or
  doprimes), I could write a function which is
  passed the function to apply using funcall

(defmacro is-greater (x y) (gt ,x ,y))