Sky Sports News has been told James will be unavailable until well into the knockout stages of the tournament as he battles to recover from a hamstring injury.
James was seen beckoning to the bench late in the 0-0 draw with Ghana on Tuesday but was unable to be replaced because Thomas Tuchel had already made all five substitutions.
- World Cup fixture schedule - your day-by-day guide
- Senegal in the last 32? England's possible routes to the final
- Download the Sky Sports app
It is not clear whether the fact he had to play on aggravated the injury.
James came into the tournament fully fit but England's two opening games in the USA were the first back-to-back matches he has managed since March.
James has not travelled with the rest of the squad to New Jersey for Saturday's final Group L game against Panama.
James is thought to be devastated by the situation but Sky Sports News has been told he has decided not to leave the camp, and instead wants to stay with the squad and fight to be fit in the hope of playing some minutes in the latter stages of the tournament.
The situation is a real problem for Tuchel, whose back-up right back, Tino Livramento, withdrew from the squad before the first game - again because of a hamstring problem.
The England boss decided to call up a centre-back - James' Chelsea team-mate, Trevoh Chalobah - in his place and now, with James absent, his right-back options are not ideal.
Djed Spence is the most obvious replacement but he has been playing left-back for Tottenham, and was selected in that position for this squad as cover for Nico O'Reilly.
Jarrel Quansah - another centre-back - can play on the right, as can Ezri Konsa, but he is currently first choice in the middle of defence.
Anderson and Rice could be rested against Panama
Elliot Anderson is still being assessed to see what part he can play against Panama after suffering a glute injury against Ghana. It is not thought to be serious, with a kick causing a dead leg.
Declan Rice, Anderson's midfield partner, did not train fully on Thursday but he did take part in the whole session on Friday.
Despite saying that Anderson and Rice are available for selection, Tuchel is considering resting one or both against Panama so they can recover fully for the knockout games.
The England head coach has some difficult calls to make throughout the side and is expected to rotate his starting XI with the knockout games in mind.
However, England have to match or better Ghana's result against Croatia on the final matchday to guarantee top spot in Group L - and with it a theoretically easier route for the rest of the tournament.