In my last post, where I wrote about sequential parameter optimization, I asked you if you did some assumptions about the underlying function when trying to find the optimal spot. (If you missed this and you’re interested in an easy introduction to sequential parameter optimization you might like to have a look here.)
But back to the assumptions. If you don’t want to do a random search, you definitely take some assumptions about the underlying function. That might be something simple, like ‘following the steepest path leads to the highest spot’ or more complex assumptions about the nature of the function, like ‘it might be polynomial’.
The former assumption can be used for a simple search strategy, the latter for building a surrogate model that hopefully resembles the function. Continue reading →