aGrUM 3.1.1
a C++ library for (probabilistic) graphical models
gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR > Class Template Reference

InformationTheory is a template class which aims at gathering the implementation of informational functions (entropy, mutual information, etc.). More...

#include <informationTheory.h>

Collaboration diagram for gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >:

Public Member Functions

 InformationTheory (INFERENCE_ENGINE< GUM_SCALAR > &engine, NodeSet X, NodeSet Y, NodeSet Z)
 InformationTheory (INFERENCE_ENGINE< GUM_SCALAR > &engine, const NodeSet &X, const NodeSet &Y)
 InformationTheory (INFERENCE_ENGINE< GUM_SCALAR > &engine, const std::vector< std::string > &Xnames, const std::vector< std::string > &Ynames)
 InformationTheory (INFERENCE_ENGINE< GUM_SCALAR > &engine, const std::vector< std::string > &Xnames, const std::vector< std::string > &Ynames, const std::vector< std::string > &Znames)
 ~InformationTheory ()
GUM_SCALAR entropyXY ()
GUM_SCALAR entropyX ()
GUM_SCALAR entropyY ()
GUM_SCALAR entropyXgivenY ()
GUM_SCALAR entropyYgivenX ()
GUM_SCALAR mutualInformationXY ()
GUM_SCALAR variationOfInformationXY ()
GUM_SCALAR entropyXgivenZ ()
GUM_SCALAR entropyYgivenZ ()
GUM_SCALAR entropyXYgivenZ ()
GUM_SCALAR entropyXgivenYZ ()
GUM_SCALAR mutualInformationXYgivenZ ()

Protected Member Functions

void makeInference_ ()

Protected Attributes

INFERENCE_ENGINE< GUM_SCALAR > & engine_
const NodeSet X_
const NodeSet Y_
const NodeSet Z_
VariableSet vX_
VariableSet vY_
VariableSet vZ_
Tensor< GUM_SCALAR > pXYZ_
Tensor< GUM_SCALAR > pXY_
Tensor< GUM_SCALAR > pXZ_
Tensor< GUM_SCALAR > pYZ_
Tensor< GUM_SCALAR > pX_
Tensor< GUM_SCALAR > pY_
Tensor< GUM_SCALAR > pZ_

Detailed Description

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
class gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >

InformationTheory is a template class which aims at gathering the implementation of informational functions (entropy, mutual information, etc.).

All these operations start with the inference of a joint distribution. Moreover, it is possible to want to condition these calculations. The inference (able to compute joint distributions) passed as an argument can therefore have previously defined evidence.

Warning
The joint distribution underlying the computations has a definition domain of exponential size depending on the number of variables passed as arguments. One must be aware of the complexity of the inference that will initially be carried out.
Note
All computations are made with std::log2
Template Parameters
INFERENCE_ENGINE: the inference engine
GUM_SCALAR: the numeric type of parameters

Definition at line 83 of file informationTheory.h.

Constructor & Destructor Documentation

◆ InformationTheory() [1/4]

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::InformationTheory ( INFERENCE_ENGINE< GUM_SCALAR > & engine,
gum::NodeSet X,
gum::NodeSet Y,
gum::NodeSet Z )

Definition at line 64 of file informationTheory_tpl.h.

70 if ((!(X_ * Y_).empty()) || (!(X_ * Z_).empty()) || (!(Z_ * Y_).empty()))
71 GUM_ERROR(OperationNotAllowed, "The intersection between the set of variables must be empty")
73 }
InformationTheory is a template class which aims at gathering the implementation of informational fun...
InformationTheory(INFERENCE_ENGINE< GUM_SCALAR > &engine, NodeSet X, NodeSet Y, NodeSet Z)
INFERENCE_ENGINE< GUM_SCALAR > & engine_
#define GUM_ERROR(type, msg)
Definition exceptions.h:76

References InformationTheory(), engine_, GUM_ERROR, makeInference_(), X_, Y_, and Z_.

Referenced by InformationTheory(), InformationTheory(), InformationTheory(), InformationTheory(), and ~InformationTheory().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ InformationTheory() [2/4]

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::InformationTheory ( INFERENCE_ENGINE< GUM_SCALAR > & engine,
const NodeSet & X,
const NodeSet & Y )

Definition at line 76 of file informationTheory_tpl.h.

Set< NodeId > NodeSet
Some typdefs and define for shortcuts ...

References InformationTheory().

Here is the call graph for this function:

◆ InformationTheory() [3/4]

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::InformationTheory ( INFERENCE_ENGINE< GUM_SCALAR > & engine,
const std::vector< std::string > & Xnames,
const std::vector< std::string > & Ynames )

Definition at line 82 of file informationTheory_tpl.h.

85 :
87 engine.model().nodeset(Xnames),
88 engine.model().nodeset(Ynames),
89 NodeSet()) {}

References InformationTheory().

Here is the call graph for this function:

◆ InformationTheory() [4/4]

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::InformationTheory ( INFERENCE_ENGINE< GUM_SCALAR > & engine,
const std::vector< std::string > & Xnames,
const std::vector< std::string > & Ynames,
const std::vector< std::string > & Znames )

Definition at line 92 of file informationTheory_tpl.h.

96 :
98 engine.model().nodeset(Xnames),
99 engine.model().nodeset(Ynames),
100 engine.model().nodeset(Znames)) {}

References InformationTheory().

Here is the call graph for this function:

◆ ~InformationTheory()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::~InformationTheory ( )

Definition at line 103 of file informationTheory_tpl.h.

103 {
105 }

References InformationTheory().

Here is the call graph for this function:

Member Function Documentation

◆ entropyX()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyX ( )

Definition at line 152 of file informationTheory_tpl.h.

152{ return pX_.entropy(); }
Tensor< GUM_SCALAR > pX_

References pX_.

◆ entropyXgivenY()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyXgivenY ( )

Definition at line 163 of file informationTheory_tpl.h.

163 {
164 return -pXY_.expectedValue([this](const gum::Instantiation& i) -> GUM_SCALAR {
165 // f(x,y)=log (p(x,y)/p(y))
166 const auto& pxy = pXY_[i];
167 if (pxy == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
168
169 const auto& py = pY_[i];
170 if (py == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
171
172 return GUM_LOG2_OR_0(pxy / py);
173 });
174 }
Tensor< GUM_SCALAR > pY_
Tensor< GUM_SCALAR > pXY_
#define GUM_LOG2_OR_0(x)
Definition math_utils.h:70

References GUM_LOG2_OR_0, pXY_, and pY_.

◆ entropyXgivenYZ()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyXgivenYZ ( )

Definition at line 258 of file informationTheory_tpl.h.

258 {
259 if (Z_.empty()) GUM_ERROR(ArgumentError, "Z has not been specified.")
261 // f(x,y,z)= -log(p(x,y,z)/p(y,z))
262 const auto& pxyz = pXYZ_[i];
263 if (pxyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
264
265 const auto& pyz = pYZ_[i];
266 if (pyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
267
268 return -GUM_LOG2_OR_0(pxyz / pyz);
269 });
270 }
Tensor< GUM_SCALAR > pXYZ_
Tensor< GUM_SCALAR > pYZ_

References GUM_ERROR, GUM_LOG2_OR_0, pXYZ_, pYZ_, and Z_.

◆ entropyXgivenZ()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyXgivenZ ( )

Definition at line 191 of file informationTheory_tpl.h.

191 {
192 if (Z_.empty()) GUM_ERROR(ArgumentError, "Z has not been specified.")
194 // f(x,z)=log (p(x,z)/p(z))
195 const auto& pxz = pXZ_[i];
196 if (pxz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
197
198 const auto& pz = pZ_[i];
199 if (pz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
200
201 return GUM_LOG2_OR_0(pxz / pz);
202 });
203 }
Tensor< GUM_SCALAR > pXZ_
Tensor< GUM_SCALAR > pZ_

References GUM_ERROR, GUM_LOG2_OR_0, pXZ_, pZ_, and Z_.

◆ entropyXY()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyXY ( )

Definition at line 158 of file informationTheory_tpl.h.

158 {
159 return pXY_.entropy();
160 }

References pXY_.

◆ entropyXYgivenZ()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyXYgivenZ ( )

Definition at line 243 of file informationTheory_tpl.h.

243 {
244 if (Z_.empty()) GUM_ERROR(ArgumentError, "Z has not been specified.")
246 // f(x,y,z)= -log(p(x,y,z)/p(z))
247 const auto& pxyz = pXYZ_[i];
248 if (pxyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
249
250 const auto& pz = pZ_[i];
251 if (pz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
252
253 return -GUM_LOG2_OR_0(pxyz / pz);
254 });
255 }

References GUM_ERROR, GUM_LOG2_OR_0, pXYZ_, pZ_, and Z_.

◆ entropyY()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyY ( )

Definition at line 155 of file informationTheory_tpl.h.

155{ return pY_.entropy(); }

References pY_.

◆ entropyYgivenX()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyYgivenX ( )

Definition at line 177 of file informationTheory_tpl.h.

177 {
178 return -pXY_.expectedValue([this](const gum::Instantiation& i) -> GUM_SCALAR {
179 // f(x,y)=log (p(x,y)/p(x))
180 const auto& pxy = pXY_[i];
181 if (pxy == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
182
183 const auto& px = pX_[i];
184 if (px == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
185
186 return GUM_LOG2_OR_0(pxy / px);
187 });
188 }

References GUM_LOG2_OR_0, pX_, and pXY_.

◆ entropyYgivenZ()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::entropyYgivenZ ( )

Definition at line 206 of file informationTheory_tpl.h.

206 {
207 if (Z_.empty()) GUM_ERROR(ArgumentError, "Z has not been specified.")
209 // f(y,z)=log (p(y,z)/p(z))
210 const auto& pyz = pYZ_[i];
211 if (pyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
212
213 const auto& pz = pZ_[i];
214 if (pz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
215
216 return GUM_LOG2_OR_0(pyz / pz);
217 });
218 }

References GUM_ERROR, GUM_LOG2_OR_0, pYZ_, pZ_, and Z_.

◆ makeInference_()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE void gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::makeInference_ ( )
protected

Definition at line 108 of file informationTheory_tpl.h.

108 {
109 vX_.clear();
110 for (const auto x: X_)
111 vX_.insert(&engine_.model().variable(x));
112
113 vY_.clear();
114 for (const auto y: Y_)
115 vY_.insert(&engine_.model().variable(y));
116
117 vZ_.clear();
118 for (const auto z: Z_)
119 vZ_.insert(&engine_.model().variable(z));
120
121 const NodeSet joint_vars = X_ + Y_ + Z_;
122 if (!engine_.isJointTarget(joint_vars)) {
123 // we check if it is not an implicit target : containng a node and some of its parent (could
124 // be better)
125 bool implicit_target = false;
126 for (const auto node: joint_vars)
127 if (engine_.model().family(node).isSupersetOrEqual(joint_vars)) {
128 implicit_target = true;
129 break;
130 }
131 if (!implicit_target) {
132 engine_.eraseAllJointTargets();
133 engine_.addJointTarget(joint_vars);
134 }
135 }
136 engine_.makeInference();
137
138 if (!Z_.empty()) {
139 pXYZ_ = engine_.jointPosterior(joint_vars);
140 pXZ_ = pXYZ_.sumIn(vX_ + vZ_);
141 pYZ_ = pXYZ_.sumIn(vY_ + vZ_);
142 pZ_ = pXZ_.sumIn(vZ_);
143 pXY_ = pXYZ_.sumIn(vX_ + vY_);
144 } else {
145 pXY_ = engine_.jointPosterior(joint_vars);
146 }
147 pX_ = pXY_.sumIn(vX_);
148 pY_ = pXY_.sumIn(vY_);
149 }

References engine_, pX_, pXY_, pXYZ_, pXZ_, pY_, pYZ_, pZ_, vX_, vY_, vZ_, X_, Y_, and Z_.

Referenced by InformationTheory().

Here is the caller graph for this function:

◆ mutualInformationXY()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::mutualInformationXY ( )

Definition at line 221 of file informationTheory_tpl.h.

221 {
222 return pXY_.expectedValue([this](const gum::Instantiation& i) -> GUM_SCALAR {
223 // f(x,y)=log (p(x,y)/p(x)p(y))
224 const auto& pxy = pXY_[i];
225 if (pxy == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
226
227 const auto& pxpy = pY_[i] * pX_[i];
228 if (pxpy == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
229
230 return GUM_LOG2_OR_0(pxy / pxpy);
231 });
232 }

References GUM_LOG2_OR_0, pX_, pXY_, and pY_.

◆ mutualInformationXYgivenZ()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::mutualInformationXYgivenZ ( )

Definition at line 273 of file informationTheory_tpl.h.

273 {
274 if (Z_.empty()) GUM_ERROR(ArgumentError, "Z has not been specified.")
276 // f(x,y)=log (p(x,y)/p(x)p(y))
277 const auto& pzpxyz = pXYZ_[i] * pZ_[i];
278 if (pzpxyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
279
280 const auto& pxzpyz = pXZ_[i] * pYZ_[i];
281 if (pxzpyz == GUM_SCALAR(0.0)) return GUM_SCALAR(0.0);
282
283 return GUM_LOG2_OR_0(pzpxyz / pxzpyz);
284 });
285 }

References GUM_ERROR, GUM_LOG2_OR_0, pXYZ_, pXZ_, pYZ_, pZ_, and Z_.

◆ variationOfInformationXY()

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFORMATION_THEORY_TEMPLATE GUM_SCALAR gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::variationOfInformationXY ( )

Definition at line 235 of file informationTheory_tpl.h.

235 {
236 return -pXY_.expectedValue([this](const gum::Instantiation& i) -> GUM_SCALAR {
237 // f(x,y)= -log(p(x)*p(y))
238 return GUM_LOG2_OR_0(pY_[i] * pX_[i]);
239 });
240 }

References GUM_LOG2_OR_0, pX_, pXY_, and pY_.

Member Data Documentation

◆ engine_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
INFERENCE_ENGINE< GUM_SCALAR >& gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::engine_
protected

Definition at line 116 of file informationTheory.h.

Referenced by InformationTheory(), and makeInference_().

◆ pX_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pX_
protected

◆ pXY_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pXY_
protected

◆ pXYZ_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pXYZ_
protected

◆ pXZ_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pXZ_
protected

Definition at line 128 of file informationTheory.h.

Referenced by entropyXgivenZ(), makeInference_(), and mutualInformationXYgivenZ().

◆ pY_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pY_
protected

◆ pYZ_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pYZ_
protected

◆ pZ_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
Tensor< GUM_SCALAR > gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::pZ_
protected

◆ vX_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
VariableSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::vX_
protected

Definition at line 122 of file informationTheory.h.

Referenced by makeInference_().

◆ vY_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
VariableSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::vY_
protected

Definition at line 123 of file informationTheory.h.

Referenced by makeInference_().

◆ vZ_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
VariableSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::vZ_
protected

Definition at line 124 of file informationTheory.h.

Referenced by makeInference_().

◆ X_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
const NodeSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::X_
protected

Definition at line 118 of file informationTheory.h.

Referenced by InformationTheory(), and makeInference_().

◆ Y_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
const NodeSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::Y_
protected

Definition at line 119 of file informationTheory.h.

Referenced by InformationTheory(), and makeInference_().

◆ Z_

template<template< typename > class INFERENCE_ENGINE, typename GUM_SCALAR>
const NodeSet gum::InformationTheory< INFERENCE_ENGINE, GUM_SCALAR >::Z_
protected

The documentation for this class was generated from the following files: