Compression ignition engine technologies have been advanced in the past decade to provide superior fuel economy and high performance. These technologies offer increased opportunities for optimizing engine calibration. Current engine calibration methods rely on deriving static tabular relationships between a set of steady-state operating points and the corresponding values of the controllable variables. While the engine is running, these values are being interpolated for each engine operating point to coordinate optimal performance criteria, e.g., fuel economy, emissions, and acceleration. These methods, however, are not efficient in capturing transient engine operation designated by common driving habits, e.g., stop-and-go driving, rapid acceleration, and braking. An alternative approach was developed recently, which makes the engine an autonomous intelligent system, namely, one capable of learning its optimal calibration for both steady-state and transient operating points in real time. Through this approach, while the engine is running the vehicle, it progressively perceives the driver’s driving style and eventually learns to operate in a manner that optimizes specified performance criteria. The major challenge to this approach is problem dimensionality when more than one controllable variable is considered. In this paper, we address this problem by proposing a decentralized learning control scheme. The scheme is evaluated through simulation of a diesel engine model, which learns the values of injection timing and variable geometry turbocharging vane position that optimize fuel economy and pollutant emissions over a segment of the FTP-75 driving cycle.

1.
Atkinson
,
C.
, and
Mott
,
G.
, 2005, “
Dynamic Model-Based Calibration Optimization: An Introduction and Application to Diesel Engines
,”
SAE World Congress
, Detroit, MI, Apr. 11–14, Paper No. SAE 2005-01-0026.
2.
Samulski
,
M. J.
, and
Jackson
,
C. C.
, 1998, “
Effects of Steady-State and Transient Operation on Exhaust Emissions From Nonroad and Highway Diesel Engines
,” SAE Transactions-Journal of Engines, Vol.
107
.
3.
Green
,
R. M.
, 2000, “
Measuring the Cylinder-to-Cylinder EGR Distribution in the Intake of a Diesel Engine During Transient Operation
,” SAE Transactions-Journal of Engines, Vol.
109
.
4.
Malikopoulos
,
A. A.
, 2008, “
Real-Time, Self-Learning Identification and Stochastic Optimal Control of Advanced Powertrain Systems
,” Ph.D. thesis, Department of Mechanical Engineering, University of Michigan, Ann Arbor.
5.
Malikopoulos
,
A. A.
,
Papalambros
,
P. Y.
, and
Assanis
,
D. N.
, 2007, “
A Learning Algorithm for Optimal Internal Combustion Engine Calibration in Real Time
,”
Proceedings of the ASME 2007 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, Las Vegas, NV, Sept. 4–7.
6.
Bush
,
R. R.
, and
Mosteler
,
F.
, 1958,
Stochastic Models for Learning
,
Wiley
,
New York
.
7.
Atkinson
,
R. C.
,
Bower
,
G. H.
, and
Crothers
,
E. J.
, 1965,
An Introduction To Mathematical Learning Theory
,
Wiley
,
New York
.
8.
Tsypkin
,
Y. Z.
, 1971,
Adaptation and Learning in Automatic Systems
,
Academic
,
New York
.
9.
Narendra
,
K. S.
, and
Wheeler
,
R. M.
, Jr.
, 1983, “
N-Player Sequential Stochastic Game With Identical Payoffs
,”
IEEE Trans. Syst. Man Cybern.
,
13
, pp.
1154
1158
. 0018-9472
10.
Srikantakumar
,
P. R.
, and
Narendra
,
K. S.
, 1982, “
A Learning Model for Routing in Telephone Networks
,”
SIAM J. Control Optim.
0363-0129,
20
, pp.
34
57
.
11.
Wu
,
H.
, 2007, “
Decentralized Iterative Learning Control for a Class of Large Scale Interconnected Dynamical Systems
,”
J. Math. Anal. Appl.
,
327
, pp.
233
245
. 0022-247X
12.
Szer
,
D.
, and
Charpillet
,
F.
, 2004, “
Improving Coordination With Communication in Multi-Agent Reinforcement Learning
,”
Proceedings of the 16th IEEE International Conference on Tools With Artificial Intelligence, ICTAI 2004
, Boca Raton, FL, pp.
436
440
.
13.
Scherrer
,
B.
, and
Charpillet
,
F.
, 2002, “
Cooperative Co-Learning: A Model-Based Approach for Solving Multi-Agent Reinforcement Problems
,”
Proceedings of the 14th IEEE International Conference on Tools With Artificial Intelligence
, Washington, DC, pp.
463
468
.
14.
Beynier
,
A.
, and
Mouaddib
,
A. -I.
, 2006, “
An Iterative Algorithm for Solving Constrained Decentralized Markov Decision Processes
,”
Proceedings of the 21st National Conference on Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conference
, Boston, MA, Paper No. AAAI-06/IAAI-06, pp.
1089
1094
.
15.
Yagan
,
D.
, and
Chen-Khong
,
T.
, 2007, “
Coordinated Reinforcement Learning for Decentralized Optimal Control
,”
Proceedings of the 2007 First IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning
, Honolulu, HI (IEEE Catalog No. 07EX1572), pp.
296
302
.
16.
Shen
,
D.
,
Chen
,
G.
,
Cruz
,
J. B.
, Jr.
,
Kwan
,
C.
, and
Kruger
,
M.
, 2007, “
An Adaptive Markov Game Model for Threat Intent Inference
,”
Proceedings of the IEEE Aerospace Conference
, Big Sky, MT, pp.
1
13
.
17.
Wheeler
,
R.
, and
Narenda
,
K.
, 1986, “
Decentralized Learning in Finite Markov Chains
,”
IEEE Trans. Autom. Control
,
31
(
6
), pp.
519
526
. 0018-9286
18.
Puterman
,
M. L.
, 2005,
Markov Decision Processes: Discrete Stochastic Dynamic Programming
, 2nd revised ed.,
Wiley-Interscience
,
New York
.
19.
Sennott
,
L. I.
, 1998,
Stochastic Dynamic Programming and the Control of Queueing Systems
, 1st ed.,
Wiley-Interscience
,
New York
.
20.
Bertsekas
,
D. P.
, and
Shreve
,
S. E.
, 2007,
Stochastic Optimal Control: The Discrete-Time Case
, 1st ed.,
Athena Scientific
,
Nashua, NH
.
21.
Bertsekas
,
D. P.
, and
Tsitsiklis
,
J. N.
, 1996,
Neuro-Dynamic Programming
(
Optimization and Neural Computation Series 3
), 1st ed.,
Athena Scientific
,
Nashua, NH
.
22.
Sutton
,
R. S.
, and
Barto
,
A. G.
, 1998,
Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning)
,
MIT
,
Cambridge, MA
.
23.
Borkar
,
V. S.
, 2000, “
A Learning Algorithm for Discrete-Time Stochastic Control
,”
Probability in the Engineering and Information Science
,
14
, pp.
243
258
. 0269-9648
24.
Malikopoulos
,
A. A.
,
Papalambros
,
P. Y.
, and
Assanis
,
D. N.
, 2007, “
A State-Space Representation Model and Learning Algorithm for Real-Time Decision-Making Under Uncertainty
,”
Proceedings of the 2007 ASME International Mechanical Engineering Congress and Exposition
, Seattle, WA, Nov. 11–15.
25.
Malikopoulos
,
A. A.
,
Assanis
,
D. N.
, and
Papalambros
,
P. Y.
, 2007, “
Real-Time, Self-Learning Optimization of Diesel Engine Calibration
,”
Proceedings of the 2007 Fall Technical Conference of the ASME Internal Combustion Engine Division
, Charleston, SC, Oct. 14–17.
26.
Heywood
,
J.
, 1988,
Internal Combustion Engine Fundamentals
, 1st ed.,
McGraw-Hill
,
New York
.
27.
TESIS
, http://www.tesis.de/en/
You do not currently have access to this content.