You are not guaranteed to find optimal solutions (although you might find local minima)

For constrained optimisation ideally you should be using Lagrange multipliers. But even then there are `abnormal` cases where the Lagrange multiplier fails to give you an optimal solution. Roughly your constraint does not have enough information to give you a unique solution (dg insufficient rank to find solutions in df = \lambda dg).

In infinite dimensions (minimising a functional) a penalty metric is an even worse an idea. My favorite example is sub-Riemannian geometry [0]. People wanted to find sub-Riemannian geodesics - special curves that could only move in restricted directions. Early on people tried to use the penalty metric idea (making it really expensive to move in certain directions), but Montgomery [0] proved that this does not necessarily give optimal solutions. The limit of an equation is not necessarily the same as the limit of the solutions. In a sense they used Lagrange multipliers to examine the situation.

Looking at abnormal optimisation problems is an interesting area of research [1]. However if a local solution is "good enough" maybe you can get away with a penalty metric.

[0] https://www.ams.org/books/surv/091/ [1] https://books.google.com.au/books?isbn=9401594384

See the literature on exact penalty methods. I believe the short of it is yes, for a large class of problems this works, but the new problem will not be necessarily easier to solve. In the case of the abs(.) function, the nonlinearity at 0 makes subgradient methods slow, and the large constant in front of the abs(.) might prove numerically unstable.

Yes, all constrained optimization problems can be converted to unconstrained problems. For example, consider:

minimize f(x) subject to x \in C

Let g(x)=f(x) if x \in C and infinity otherwise. Then

minimize g(x)

has the same solution as the original constrained problem. However, this conversion often conceals some of the structure of the original problem which can be exploited to solve the problem more efficiently.

Solving point problems as you have called them typically involves solving a sequence of least squares problems, which are simple to reason about and computationally efficient to solve. Solving a calculus of variations problem typically involves solving an integral or partial differential equation. Although there are theoretical similarities in practice they are pretty different.

To add to the answers already given, this is indeed a viable option to solve some problems, see e.g. the barrier method for linear programming. But then the constant 10^30 is gradually increased for numerical reasons.

I assume you mean norm rather than absolute value because absolute value would give a vector result?

But yes, but this is generally unhelpful since in general the behavior at a given point would give you almost no information regarding behavior at a nearby point. It's a bit like doing the reverse by turning the objective into a feasibility test. e.g. min f(x) is like min 0 : {f(x) <= f(x') for all x'}... you can do it, but it's not really helpful.

well you're technically solving a different problem in the second case, you won't always get the same answer and sometimes adding that can mess up your solver. However a very common method for handling tricky constraints is by "relaxing" them, by moving g(x) into the cost function, exactly as you propose.

I think trajectory optimization can still be viewed as a point optimization, and it's helpful to understand the fundamentals before jumping into applications.

It could be a time factor. That was published in 2006. "An introduction to optimization and to the calculus of variations" is published in 2017 https://www.ljll.math.upmc.fr/mazari/documents/Cours_Pristin...

a) this is not the same problem. The second equation you suggest could have a different minimum than the first, e.g. if f(x) goes to negative infinity but only when g(x) != 0.

b) because this is a much more complex topic, for which you need to know optimization first anyways.