He should leave out the f(x). It has no meaning here.
It should be
2) 15. Let f(a,b) be defined as a^2+b^2-2, which of the following CANNOT be the value of f(a,b)?
Is it 2?
What kind of numbers are a and b? Integers? What other constraints are put on them?
In the US there are 1 cent coins, 5 cent coins, 10 cent coins, 25 cent coins and 50 cent coins. A person goes to the store and hands the cashier 1 dollar ( worth 100 cents ). He asks her to change the dollar into coins.
One way for her to change his dollar into coins would be one 50 cent coin, one 25 cent coin, one 10 cent coin, one 5 cent coin and ten one cent coins. The math question is: how many ways can she give him change.
This is a pretty sophisticated math problem, many will be unable to answer, many more will be unable to compute the answer even if they could set it up. To even a beginning programmer this problem is trivial. A case of code being much easier than math.
There are many other such cases.
They are used for lots of things but two things come to mind quickly, sums and integrals.