假设S是$ \ mathbb {R} ^ n $中的非空开放集，则$ f：S \ rightarrow \ mathbb {R} $在$ \ hat {x} \ in S $中可微分存在一个称为梯度矢量的矢量$ \ bigtriangledown f \ left(\ hat {x} \ right)$和一个函数$ \ alpha：\ mathbb {R} ^ n \ rightarrow \ mathbb {R} $

$ f \ left(x \ right)= f \ left(\ hat {x} \ right)+ \ bigtriangledown f \ left(\ hat {x} \ right)^ T \ left(x- \ hat {x} \右)+ \左\ | x = \ hat {x} \ right \ | \ alpha \ left(\ hat {x}，x- \ hat {x} \ right)，\ forall x \ in S $其中

$ \ alpha \ left(\ hat {x}，x- \ hat {x} \ right)\ rightarrow 0 \ bigtriangledown f \ left(\ hat {x} \ right)= \ left [\ frac {\ partial f} {\ partial x_1} \ frac {\ partial f} {\ partial x_2} … \ frac {\ partial f} {\ partial x_n} \ right] _ {x = \ hat {x}} ^ {T} $

定理

令S为$ \ mathbb {R} ^ n $中的非空开放凸集，并使$ f：S \ rightarrow \ mathbb {R} $在S上可微。然后，当且仅当$为x_1，x_2 \ in S，\ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)\ leq f \ left(x_1 \ right)-f \ left(x_2 \ right)$

证明

令f为凸函数。即对于$ x_1，x_2 \ in S，\ lambda \ in \ left(0，1 \ right)$

$ f \ left [\ lambda x_1 + \ left(1- \ lambda \ right)x_2 \ right] \ leq \ lambda f \ left(x_1 \ right)+ \ left(1- \ lambda \ right)f \ left(x_2 \ right)$

$ \ Rightarrow f \ left [\ lambda x_1 + \ left(1- \ lambda \ right)x_2 \ right] \ leq \ lambda \ left(f \ left(x_1 \ right)-f \ left(x_2 \ right)\ right )+ f \左(x_2 \ right)$

$ \ Rightarrow \ lambda \ left(f \ left(x_1 \ right)-f \ left(x_2 \ right)\ right)\ geq f \ left(x_2 + \ lambda \ left(x_1-x_2 \ right)\ right)- f \ left(x_2 \ right)$

$ \ Rightarrow \ lambda \ left(f \ left(x_1 \ right)-f \ left(x_2 \ right)\ right)\ geq f \ left(x_2 \ right)+ \ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)\ lambda + $

$ \左\ | \ lambda \ left(x_1-x_2 \ right)\ right \ | \ alpha \ left(x_2，\ lambda \ left(x_1-x_2 \ right)-f \ left(x_2 \ right)\ right)$

其中$ \ alpha \ left(x_2，\ lambda \ left(x_1-x_2 \ right)\ right)\ rightarrow 0 $ as $ \ lambda \ rightarrow 0 $

在两侧除以$ \ lambda $，我们得到-

$ f \ left(x_1 \ right)-f \ left(x_2 \ right)\ geq \ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)$

交谈

设$ x_1，x_2 \ in S，\ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)\ leq f \ left(x_1 \ right)-f \ left(x_2 \ right) $

为了证明f是凸的。

由于S为凸面，$ x_3 = \ lambda x_1 + \ left(1- \ lambda \ right)x_2 \ in S，\ lambda \ in \ left(0，1 \ right)$

由于$ x_1，x_3 \ in S $，因此

$ f \ left(x_1 \ right)-f \ left(x_3 \ right)\ geq \ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_1 -x_3 \ right)$

$ \ Rightarrow f \ left(x_1 \ right)-f \ left(x_3 \ right)\ geq \ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_1-\ lambda x_1- \ left(1- \ lambda \ right)x_2 \ right)$

$ \ Rightarrow f \ left(x_1 \ right)-f \ left(x_3 \ right)\ geq \ left(1- \ lambda \ right)\ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_1-x_2 \ right)$

由于$ x_2，x_3 \ in S $

$ f \ left(x_2 \ right)-f \ left(x_3 \ right)\ geq \ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_2-x_3 \ right)$

$ \ Rightarrow f \ left(x_2 \ right)-f \ left(x_3 \ right)\ geq \ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_2- \ lambda x_1- \ left(1- \ lambda \ right)x_2 \ right)$

$ \ Rightarrow f \ left(x_2 \ right)-f \ left(x_3 \ right)\ geq \ left(-\ lambda \ right)\ bigtriangledown f \ left(x_3 \ right)^ T \ left(x_1-x_2 \右)$

因此，结合以上等式，我们得到-

$ \ lambda \ left(f \ left(x_1 \ right)-f \ left(x_3 \ right)\ right)+ \ left(1- \ lambda \ right)\ left(f \ left(x_2 \ right)-f \ left(x_3 \ right)\ right)\ geq 0 $

$ \ Rightarrow f \ left(x_3 \ right)\ leq \ lambda f \ left(x_1 \ right)+ \ left(1- \ lambda \ right)f \ left(x_2 \ right)$

定理

设S为$ \ mathbb {R} ^ n $中的非空开放凸集，并让$ f：S \ rightarrow \ mathbb {R} $在S上可微，那么当且仅当任何$ x_1，x_2 \ in S，\ left(\ bigtriangledown f \ left(x_2 \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(x_2-x_1 \ right)\ geq 0 $

证明

令f为凸函数，然后使用先前的定理-

$ \ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)\ leq f \ left(x_1 \ right)-f \ left(x_2 \ right)$和

$ \ bigtriangledown f \ left(x_1 \ right)^ T \ left(x_2-x_1 \ right)\ leq f \ left(x_2 \ right)-f \ left(x_1 \ right)$

将以上两个方程相加，我们得到-

$ \ bigtriangledown f \ left(x_2 \ right)^ T \ left(x_1-x_2 \ right)+ \ bigtriangledown f \ left(x_1 \ right)^ T \ left(x_2-x_1 \ right)\ leq 0 $

$ \ Rightarrow \ left(\ bigtriangledown f \ left(x_2 \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(x_1-x_2 \ right)\ leq 0 $

$ \ Rightarrow \ left(\ bigtriangledown f \ left(x_2 \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(x_2-x_1 \ right)\ geq 0 $

交谈

让任何$ x_1，x_2 \ in S，\ left(\ bigtriangledown f \ left(x_2 \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(x_2-x_1 \ right)\ geq 0 $

为了证明f是凸的。

令$ x_1，x_2 \ in S $，因此根据均值定理$ \ frac {f \ left(x_1 \ right)-f \ left(x_2 \ right)} {x_1-x_2} = \ bigtriangledown f \ left( x \ right)，x \ in \ left(x_1-x_2 \ right)\ Rightarrow x = \ lambda x_1 + \ left(1- \ lambda \ right)x_2 $因为S是一个凸集。

$ \ Rightarrow f \ left(x_1 \ right)-f \ left(x_2 \ right)= \ left(\ bigtriangledown f \ left(x \ right)^ T \ right)\ left(x_1-x_2 \ right)$

对于$ x，x_1 $，我们知道-

$ \ left(\ bigtriangledown f \ left(x \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(x-x_1 \ right)\ geq 0 $

$ \ Rightarrow \ left(\ bigtriangledown f \ left(x \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(\ lambda x_1 + \ left(1- \ lambda \ right)x_2- x_1 \ right)\ geq 0 $

$ \ Rightarrow \ left(\ bigtriangledown f \ left(x \ right)-\ bigtriangledown f \ left(x_1 \ right)\ right)^ T \ left(1- \ lambda \ right)\ left(x_2-x_1 \ right )\ geq 0 $

$ \ Rightarrow \ bigtriangledown f \ left(x \ right)^ T \ left(x_2-x_1 \ right)\ geq \ bigtriangledown f \ left(x_1 \ right)^ T \ left(x_2-x_1 \ right)$

结合以上方程，我们得到-

$ \ Rightarrow \ bigtriangledown f \ left(x_1 \ right)^ T \ left(x_2-x_1 \ right)\ leq f \ left(x_2 \ right)-f \ left(x_1 \ right)$

因此，使用最后一个定理，f是一个凸函数。

两次微分函数

假设S是$ \ mathbb {R} ^ n $的非空子集，并且让$ f：S \ rightarrow \ mathbb {R} $，则f在S中的$ \ bar {x} \处被二次微分。 $如果存在向量$ \ bigtriangledown f \ left(\ bar {x} \ right)，一个\：nXn $矩阵$ H \ left(x \ right)$(称为Hessian矩阵)和一个函数$ \ alpha： \ mathbb {R} ^ n \ rightarrow \ mathbb {R} $这样$ f \ left(x \ right)= f \ left(\ bar {x} + x- \ bar {x} \ right)= f \左(\ bar {x} \ right)+ \ bigtriangledown f \左(\ bar {x} \ right)^ T \ left(x- \ bar {x} \ right)+ \ frac {1} {2} \左(x- \ bar {x} \ right)H \左(\ bar {x} \ right)\左(x- \ bar {x} \ right)$

其中$ \ alpha \ left(\ bar {x}，x- \ bar {x} \ right)\ rightarrow Oasx \ rightarrow \ bar {x} $