Dual-Sourcing Neural Network Controller

Rather than adopting computation-heavy approaches such as dynamic programming, we can parameterize actions using a neural network. The optimization process is illustrated schematically in the figure below. The states \(\{\mathbf{s}_t^{(j)}\}\) (\(j \in \{1, \dots, M\}\)), which evolve according to the underlying discrete-time dynamics, are used as inputs to a neural network. This network is trained on simulated data generated by the sourcing models to produce actions that minimize the expected cost per period.

Neural-network optimization process

For further details, see Böttcher, Asikis, and Fragkos (2023).

Key Concepts

  • Neural Network: Instead of relying on fixed rules like the capped dual index, actions are parameterized through a neural network that learns effective order policies based on training data.

  • Discrete-Time Dynamics: The system evolves over discrete time intervals, and the controller must optimize actions at each step to reduce long-term costs.

  • Training and Optimization: The neural network aims to minimize the expected per-period cost, balancing factors such as lead time, holding costs, shortage costs, past orders and inventory level. It is trained using simulated data provided by the sourcing mode to learn optimal policies for varying demand patterns and lead times.

For further details, see Böttcher, Asikis, and Fragkos (2023).

Example Usage

We now present one example to demonstrate how the DualSourcingNeuralController can be called, trained, and evaluated in idinn.

>>> import torch
>>> from idinn.sourcing_model import DualSourcingModel
>>> from idinn.dual_controller import DualSourcingNeuralController
>>> from idinn.demand import UniformDemand
>>> from torch.utils.tensorboard import SummaryWriter

>>> dual_sourcing_model = DualSourcingModel(
...     regular_lead_time=2,
...     expedited_lead_time=0,
...     regular_order_cost=0,
...     expedited_order_cost=20,
...     holding_cost=5,
...     shortage_cost=495,
...     batch_size=256,
...     init_inventory=6,
...     demand_generator=UniformDemand(low=0, high=4),
... )
>>> controller_neural = DualSourcingNeuralController(
...     hidden_layers=[128, 64, 32, 16, 8, 4],
...     activation=torch.nn.CELU(alpha=1)
... )
>>> controller_neural.fit(
...     sourcing_model=dual_sourcing_model,
...     sourcing_periods=100,
...     validation_sourcing_periods=1000,
...     epochs=2000,
...     tensorboard_writer=SummaryWriter(comment="dual"),
...     seed=123,
... )
>>> # Avg. cost near 24
>>> avg_cost = controller_neural.get_average_cost(dual_sourcing_model, sourcing_periods=1000)
>>> print(f"Average cost: {avg_cost:.2f}")
Average cost: 24.00

Adjusting parameters such as batch_size, init_inventory, and epochs can improve the learning of sourcing policies. It may also be helpful to try out different neural-network structures.

For a given controller, orders can be predicted as follows.

>>> controller_neural.predict(current_inventory=10, past_regular_orders=[1, 1], past_expedited_orders=None)

If the regular and expedited lead-time values are greater than 0, one has to specify the corresponding past_regular_orders and past_expedited_orders.

References

  • Böttcher, L., Asikis, T., & Fragkos, I. (2023). Control of dual-sourcing inventory systems using recurrent neural networks. INFORMS Journal on Computing, 35(6), 1308–1328.