[v2,0/7] PCI: Solve two bridge window sizing issues

  Hi all,

Here's a series that contains two fixes to PCI bridge window sizing
algorithm. Together, they should enable remove & rescan cycle to work
for a PCI bus that has PCI devices with optional resources and/or
disparity in BAR sizes.

For the second fix, I chose to expose find_empty_resource_slot() from
kernel/resource.c because it should increase accuracy of the cannot-fit
decision (currently that function is called find_resource()). In order
to do that sensibly, a few improvements seemed in order to make its
interface and name of the function sane before exposing it. Thus, the
few extra patches on resource side.

Unfortunately I don't have a reason to suspect these would help with
the issues related to the currently ongoing resource regression
thread [1].

[1] https://lore.kernel.org/linux-pci/ZXpaNCLiDM+Kv38H@marvin.atrad.com.au/

v2:
- Add "typedef" to kerneldoc to get correct formatting
- Use RESOURCE_SIZE_MAX instead of literal
- Remove unnecessary checks for io{port/mem}_resource
- Apply a few style tweaks from Andy

Ilpo Järvinen (7):
  PCI: Fix resource double counting on remove & rescan
  resource: Rename find_resource() to find_empty_resource_slot()
  resource: Document find_empty_resource_slot() and resource_constraint
  resource: Use typedef for alignf callback
  resource: Handle simple alignment inside __find_empty_resource_slot()
  resource: Export find_empty_resource_slot()
  PCI: Relax bridge window tail sizing rules

 drivers/pci/bus.c       | 10 ++----
 drivers/pci/setup-bus.c | 80 +++++++++++++++++++++++++++++++++++++----
 include/linux/ioport.h  | 44 ++++++++++++++++++++---
 include/linux/pci.h     |  5 +--
 kernel/resource.c       | 68 ++++++++++++++++-------------------
 5 files changed, 148 insertions(+), 59 deletions(-)
  

Hi Ilpo,

On Thu, Dec 28, 2023 at 06:57:00PM +0200, Ilpo Järvinen wrote:
> Hi all,
> 
> Here's a series that contains two fixes to PCI bridge window sizing
> algorithm. Together, they should enable remove & rescan cycle to work
> for a PCI bus that has PCI devices with optional resources and/or
> disparity in BAR sizes.
> 
> For the second fix, I chose to expose find_empty_resource_slot() from
> kernel/resource.c because it should increase accuracy of the cannot-fit
> decision (currently that function is called find_resource()). In order
> to do that sensibly, a few improvements seemed in order to make its
> interface and name of the function sane before exposing it. Thus, the
> few extra patches on resource side.
> 
> Unfortunately I don't have a reason to suspect these would help with
> the issues related to the currently ongoing resource regression
> thread [1].
> 
> [1] https://lore.kernel.org/linux-pci/ZXpaNCLiDM+Kv38H@marvin.atrad.com.au/
> 
> v2:
> - Add "typedef" to kerneldoc to get correct formatting
> - Use RESOURCE_SIZE_MAX instead of literal
> - Remove unnecessary checks for io{port/mem}_resource
> - Apply a few style tweaks from Andy
> 
> Ilpo Järvinen (7):
>   PCI: Fix resource double counting on remove & rescan
>   resource: Rename find_resource() to find_empty_resource_slot()
>   resource: Document find_empty_resource_slot() and resource_constraint
>   resource: Use typedef for alignf callback
>   resource: Handle simple alignment inside __find_empty_resource_slot()
>   resource: Export find_empty_resource_slot()
>   PCI: Relax bridge window tail sizing rules

Thanks for doing this! :)

All look good to me,

Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
  

On Thu, 28 Dec 2023 18:57:00 +0200
Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:

> Hi all,
> 
> Here's a series that contains two fixes to PCI bridge window sizing
> algorithm. Together, they should enable remove & rescan cycle to work
> for a PCI bus that has PCI devices with optional resources and/or
> disparity in BAR sizes.
> 
> For the second fix, I chose to expose find_empty_resource_slot() from
> kernel/resource.c because it should increase accuracy of the cannot-fit
> decision (currently that function is called find_resource()). In order
> to do that sensibly, a few improvements seemed in order to make its
> interface and name of the function sane before exposing it. Thus, the
> few extra patches on resource side.
> 
> Unfortunately I don't have a reason to suspect these would help with
> the issues related to the currently ongoing resource regression
> thread [1].

Jonathan,
can you test this series on affected machine with broken kernel to see if
it's of any help in your case?

> 
> [1] https://lore.kernel.org/linux-pci/ZXpaNCLiDM+Kv38H@marvin.atrad.com.au/
> 
> v2:
> - Add "typedef" to kerneldoc to get correct formatting
> - Use RESOURCE_SIZE_MAX instead of literal
> - Remove unnecessary checks for io{port/mem}_resource
> - Apply a few style tweaks from Andy
> 
> Ilpo Järvinen (7):
>   PCI: Fix resource double counting on remove & rescan
>   resource: Rename find_resource() to find_empty_resource_slot()
>   resource: Document find_empty_resource_slot() and resource_constraint
>   resource: Use typedef for alignf callback
>   resource: Handle simple alignment inside __find_empty_resource_slot()
>   resource: Export find_empty_resource_slot()
>   PCI: Relax bridge window tail sizing rules
> 
>  drivers/pci/bus.c       | 10 ++----
>  drivers/pci/setup-bus.c | 80 +++++++++++++++++++++++++++++++++++++----
>  include/linux/ioport.h  | 44 ++++++++++++++++++++---
>  include/linux/pci.h     |  5 +--
>  kernel/resource.c       | 68 ++++++++++++++++-------------------
>  5 files changed, 148 insertions(+), 59 deletions(-)
>
  

On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> On Thu, 28 Dec 2023 18:57:00 +0200
> Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> 
> > Hi all,
> > 
> > Here's a series that contains two fixes to PCI bridge window sizing
> > algorithm. Together, they should enable remove & rescan cycle to work
> > for a PCI bus that has PCI devices with optional resources and/or
> > disparity in BAR sizes.
> > 
> > For the second fix, I chose to expose find_empty_resource_slot() from
> > kernel/resource.c because it should increase accuracy of the cannot-fit
> > decision (currently that function is called find_resource()). In order
> > to do that sensibly, a few improvements seemed in order to make its
> > interface and name of the function sane before exposing it. Thus, the
> > few extra patches on resource side.
> > 
> > Unfortunately I don't have a reason to suspect these would help with
> > the issues related to the currently ongoing resource regression
> > thread [1].
> 
> Jonathan,
> can you test this series on affected machine with broken kernel to see if
> it's of any help in your case?

Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
still on leave this week, and when at work I only have physical access to
the machine concerned on Thursdays at present.

Which kernel would you prefer I apply the series to?

Regards
  jonathan
  

On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > On Thu, 28 Dec 2023 18:57:00 +0200
> > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > 
> > > Hi all,
> > > 
> > > Here's a series that contains two fixes to PCI bridge window sizing
> > > algorithm. Together, they should enable remove & rescan cycle to work
> > > for a PCI bus that has PCI devices with optional resources and/or
> > > disparity in BAR sizes.
> > > 
> > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > decision (currently that function is called find_resource()). In order
> > > to do that sensibly, a few improvements seemed in order to make its
> > > interface and name of the function sane before exposing it. Thus, the
> > > few extra patches on resource side.
> > > 
> > > Unfortunately I don't have a reason to suspect these would help with
> > > the issues related to the currently ongoing resource regression
> > > thread [1].
> > 
> > Jonathan,
> > can you test this series on affected machine with broken kernel to see if
> > it's of any help in your case?
> 
> Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> still on leave this week, and when at work I only have physical access to
> the machine concerned on Thursdays at present.
> 
> Which kernel would you prefer I apply the series to?

I was very short of time today but I did apply the above series to the
5.15.y branch (since I had this source available), resulting in version
5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
bisect reset, so the resulting kernel was not correctly built.  It booted
but thought it was a different version and therefore none of the modules
could be found.  As a result, the test is invalid.

I will try again in a week when I next have physical access to the system. 
Apologies for the delay.  In the meantime, if there's a specific kernel I
should apply the patch series against please let me know.  As I understand
it, you want it applied to one of the kernels which failed, making 5.15.y
(for y < 145) a reasonable choice.

Regards
  jonathan
  

On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:
> On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > 
> > > > Hi all,
> > > > 
> > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > disparity in BAR sizes.
> > > > 
> > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > decision (currently that function is called find_resource()). In order
> > > > to do that sensibly, a few improvements seemed in order to make its
> > > > interface and name of the function sane before exposing it. Thus, the
> > > > few extra patches on resource side.
> > > > 
> > > > Unfortunately I don't have a reason to suspect these would help with
> > > > the issues related to the currently ongoing resource regression
> > > > thread [1].
> > > 
> > > Jonathan,
> > > can you test this series on affected machine with broken kernel to see if
> > > it's of any help in your case?
> > 
> > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > still on leave this week, and when at work I only have physical access to
> > the machine concerned on Thursdays at present.
> > 
> > Which kernel would you prefer I apply the series to?
> 
> I was very short of time today but I did apply the above series to the
> 5.15.y branch (since I had this source available), resulting in version
> 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> bisect reset, so the resulting kernel was not correctly built.  It booted
> but thought it was a different version and therefore none of the modules
> could be found.  As a result, the test is invalid.
> 
> I will try again in a week when I next have physical access to the system. 
> Apologies for the delay.  In the meantime, if there's a specific kernel I
> should apply the patch series against please let me know.  As I understand
> it, you want it applied to one of the kernels which failed, making 5.15.y
> (for y < 145) a reasonable choice.

I did a "make clean" to reset the source tree and recompiled.  However, it
errored out:

  drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
  drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared

This was with the patch series applied against 5.15.141.  It seems the patch
targets a kernel that's too far removed from 5.15.x.

Which kernel would you like me to apply the patch series to and test?

Regards
  jonathan
  

On Thu, 18 Jan 2024, Jonathan Woithe wrote:

> On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:
> > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > 
> > > > > Hi all,
> > > > > 
> > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > disparity in BAR sizes.
> > > > > 
> > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > decision (currently that function is called find_resource()). In order
> > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > few extra patches on resource side.
> > > > > 
> > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > the issues related to the currently ongoing resource regression
> > > > > thread [1].
> > > > 
> > > > Jonathan,
> > > > can you test this series on affected machine with broken kernel to see if
> > > > it's of any help in your case?
> > > 
> > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > still on leave this week, and when at work I only have physical access to
> > > the machine concerned on Thursdays at present.
> > > 
> > > Which kernel would you prefer I apply the series to?
> > 
> > I was very short of time today but I did apply the above series to the
> > 5.15.y branch (since I had this source available), resulting in version
> > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > bisect reset, so the resulting kernel was not correctly built.  It booted
> > but thought it was a different version and therefore none of the modules
> > could be found.  As a result, the test is invalid.
> > 
> > I will try again in a week when I next have physical access to the system. 
> > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > should apply the patch series against please let me know.  As I understand
> > it, you want it applied to one of the kernels which failed, making 5.15.y
> > (for y < 145) a reasonable choice.
> 
> I did a "make clean" to reset the source tree and recompiled.  However, it
> errored out:
> 
>   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
>   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> 
> This was with the patch series applied against 5.15.141.  It seems the patch
> targets a kernel that's too far removed from 5.15.x.
> 
> Which kernel would you like me to apply the patch series to and test?

Two argument version of pci_bus_for_each_resource() is quite new (so 
either 6.6 or 6.7). If want to attempt to compile in 5.15.x, you need 
this:

include/linux/limits.h:#define RESOURCE_SIZE_MAX        ((resource_size_t)~0)

And to add one extra argument into pci_bus_for_each_resource(bus, r) in 
pbus_upstream_assigned_limit():

	...
	while ((bus = bus->parent)) {
+		unsigned int i;
		if (pci_is_root_bus(bus))
			break;

-		pci_bus_for_each_resource(bus, r) {
+		pci_bus_for_each_resource(bus, r, i) {

Note I've written this "patch" by hand inline so patch command cannot 
apply it but you need to edit those in.
  

On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:
> On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:
> > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > 
> > > > > Hi all,
> > > > > 
> > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > disparity in BAR sizes.
> > > > > 
> > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > decision (currently that function is called find_resource()). In order
> > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > few extra patches on resource side.
> > > > > 
> > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > the issues related to the currently ongoing resource regression
> > > > > thread [1].
> > > > 
> > > > Jonathan,
> > > > can you test this series on affected machine with broken kernel to see if
> > > > it's of any help in your case?
> > > 
> > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > still on leave this week, and when at work I only have physical access to
> > > the machine concerned on Thursdays at present.
> > > 
> > > Which kernel would you prefer I apply the series to?
> > 
> > I was very short of time today but I did apply the above series to the
> > 5.15.y branch (since I had this source available), resulting in version
> > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > bisect reset, so the resulting kernel was not correctly built.  It booted
> > but thought it was a different version and therefore none of the modules
> > could be found.  As a result, the test is invalid.
> > 
> > I will try again in a week when I next have physical access to the system. 
> > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > should apply the patch series against please let me know.  As I understand
> > it, you want it applied to one of the kernels which failed, making 5.15.y
> > (for y < 145) a reasonable choice.
> 
> I did a "make clean" to reset the source tree and recompiled.  However, it
> errored out:
> 
>   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
>   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> 
> This was with the patch series applied against 5.15.141.  It seems the patch
> targets a kernel that's too far removed from 5.15.x.
> 
> Which kernel would you like me to apply the patch series to and test?

The rule of thumb is to test against latest vanilla (as of today v6.7).
Also makes sense to test against Linux Next. The v5.15 is way too old for
a new code.
  

On Sun, Jan 21, 2024 at 02:54:22PM +0200, Andy Shevchenko wrote:
> On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:
> > On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:
> > > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> > > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > 
> > > > > > Hi all,
> > > > > > 
> > > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > > disparity in BAR sizes.
> > > > > > 
> > > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > > decision (currently that function is called find_resource()). In order
> > > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > > few extra patches on resource side.
> > > > > > 
> > > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > > the issues related to the currently ongoing resource regression
> > > > > > thread [1].
> > > > > 
> > > > > Jonathan,
> > > > > can you test this series on affected machine with broken kernel to see if
> > > > > it's of any help in your case?
> > > > 
> > > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > > still on leave this week, and when at work I only have physical access to
> > > > the machine concerned on Thursdays at present.
> > > > 
> > > > Which kernel would you prefer I apply the series to?
> > > 
> > > I was very short of time today but I did apply the above series to the
> > > 5.15.y branch (since I had this source available), resulting in version
> > > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > > bisect reset, so the resulting kernel was not correctly built.  It booted
> > > but thought it was a different version and therefore none of the modules
> > > could be found.  As a result, the test is invalid.
> > > 
> > > I will try again in a week when I next have physical access to the system. 
> > > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > > should apply the patch series against please let me know.  As I understand
> > > it, you want it applied to one of the kernels which failed, making 5.15.y
> > > (for y < 145) a reasonable choice.
> > 
> > I did a "make clean" to reset the source tree and recompiled.  However, it
> > errored out:
> > 
> >   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
> >   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> > 
> > This was with the patch series applied against 5.15.141.  It seems the patch
> > targets a kernel that's too far removed from 5.15.x.
> > 
> > Which kernel would you like me to apply the patch series to and test?
> 
> The rule of thumb is to test against latest vanilla (as of today v6.7).
> Also makes sense to test against Linux Next. The v5.15 is way too old for
> a new code.

Thanks, and understood.  In this case the request from Igor was 

    can you test this series on affected machine with broken kernel to see if
    it's of any help in your case?

The latest vanilla kernel (6.7) has (AFAIK) had the offending commit
reverted, so it's not a "broken" kernel in this respect.  Therefore, if I've
understood the request correctly, working with that kernel won't produce the
desired test.

Regards
  jonathan
  

On Mon, 22 Jan 2024, Jonathan Woithe wrote:

> On Sun, Jan 21, 2024 at 02:54:22PM +0200, Andy Shevchenko wrote:
> > On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:
> > > On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:
> > > > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:
> > > > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:
> > > > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > > 
> > > > > > > Hi all,
> > > > > > > 
> > > > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > > > disparity in BAR sizes.
> > > > > > > 
> > > > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > > > decision (currently that function is called find_resource()). In order
> > > > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > > > few extra patches on resource side.
> > > > > > > 
> > > > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > > > the issues related to the currently ongoing resource regression
> > > > > > > thread [1].
> > > > > > 
> > > > > > Jonathan,
> > > > > > can you test this series on affected machine with broken kernel to see if
> > > > > > it's of any help in your case?
> > > > > 
> > > > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > > > still on leave this week, and when at work I only have physical access to
> > > > > the machine concerned on Thursdays at present.
> > > > > 
> > > > > Which kernel would you prefer I apply the series to?
> > > > 
> > > > I was very short of time today but I did apply the above series to the
> > > > 5.15.y branch (since I had this source available), resulting in version
> > > > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > > > bisect reset, so the resulting kernel was not correctly built.  It booted
> > > > but thought it was a different version and therefore none of the modules
> > > > could be found.  As a result, the test is invalid.
> > > > 
> > > > I will try again in a week when I next have physical access to the system. 
> > > > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > > > should apply the patch series against please let me know.  As I understand
> > > > it, you want it applied to one of the kernels which failed, making 5.15.y
> > > > (for y < 145) a reasonable choice.
> > > 
> > > I did a "make clean" to reset the source tree and recompiled.  However, it
> > > errored out:
> > > 
> > >   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
> > >   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> > > 
> > > This was with the patch series applied against 5.15.141.  It seems the patch
> > > targets a kernel that's too far removed from 5.15.x.
> > > 
> > > Which kernel would you like me to apply the patch series to and test?
> > 
> > The rule of thumb is to test against latest vanilla (as of today v6.7).
> > Also makes sense to test against Linux Next. The v5.15 is way too old for
> > a new code.
> 
> Thanks, and understood.  In this case the request from Igor was 
> 
>     can you test this series on affected machine with broken kernel to see if
>     it's of any help in your case?
> 
> The latest vanilla kernel (6.7) has (AFAIK) had the offending commit
> reverted, so it's not a "broken" kernel in this respect.  Therefore, if I've
> understood the request correctly, working with that kernel won't produce the
> desired test.

Well, you can revert the revert again to get back to the broken state.
  

On Mon, 22 Jan 2024 14:37:32 +0200 (EET)
Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:

> On Mon, 22 Jan 2024, Jonathan Woithe wrote:
> 
> > On Sun, Jan 21, 2024 at 02:54:22PM +0200, Andy Shevchenko wrote:  
> > > On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:  
> > > > On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:  
> > > > > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:  
> > > > > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:  
> > > > > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > > >   
> > > > > > > > Hi all,
> > > > > > > > 
> > > > > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > > > > disparity in BAR sizes.
> > > > > > > > 
> > > > > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > > > > decision (currently that function is called find_resource()). In order
> > > > > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > > > > few extra patches on resource side.
> > > > > > > > 
> > > > > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > > > > the issues related to the currently ongoing resource regression
> > > > > > > > thread [1].  
> > > > > > > 
> > > > > > > Jonathan,
> > > > > > > can you test this series on affected machine with broken kernel to see if
> > > > > > > it's of any help in your case?  
> > > > > > 
> > > > > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > > > > still on leave this week, and when at work I only have physical access to
> > > > > > the machine concerned on Thursdays at present.
> > > > > > 
> > > > > > Which kernel would you prefer I apply the series to?  
> > > > > 
> > > > > I was very short of time today but I did apply the above series to the
> > > > > 5.15.y branch (since I had this source available), resulting in version
> > > > > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > > > > bisect reset, so the resulting kernel was not correctly built.  It booted
> > > > > but thought it was a different version and therefore none of the modules
> > > > > could be found.  As a result, the test is invalid.
> > > > > 
> > > > > I will try again in a week when I next have physical access to the system. 
> > > > > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > > > > should apply the patch series against please let me know.  As I understand
> > > > > it, you want it applied to one of the kernels which failed, making 5.15.y
> > > > > (for y < 145) a reasonable choice.  
> > > > 
> > > > I did a "make clean" to reset the source tree and recompiled.  However, it
> > > > errored out:
> > > > 
> > > >   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
> > > >   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> > > > 
> > > > This was with the patch series applied against 5.15.141.  It seems the patch
> > > > targets a kernel that's too far removed from 5.15.x.
> > > > 
> > > > Which kernel would you like me to apply the patch series to and test?  
> > > 
> > > The rule of thumb is to test against latest vanilla (as of today v6.7).
> > > Also makes sense to test against Linux Next. The v5.15 is way too old for
> > > a new code.  
> > 
> > Thanks, and understood.  In this case the request from Igor was 
> > 
> >     can you test this series on affected machine with broken kernel to see if
> >     it's of any help in your case?
> > 
> > The latest vanilla kernel (6.7) has (AFAIK) had the offending commit
> > reverted, so it's not a "broken" kernel in this respect.  Therefore, if I've
> > understood the request correctly, working with that kernel won't produce the
> > desired test.  
> 
> Well, you can revert the revert again to get back to the broken state.

either this or just a hand patching as Ilpo has suggested earlier
would do.

There is non zero chance that this series might fix issues
Jonathan is facing. i.e. failed resource reallocation which
offending patches trigger. There are 2 different issues here,
 * 1st unwanted reallocation - it should happen but well that how current code works
 * 2nd failed reallocation - seemingly matches what this series  is trying to fix
   and if it doesn't help we would need to dig some more in this direction
   as well to figure out why it fails.
  

On Mon, Jan 22, 2024 at 02:45:20PM +0100, Igor Mammedov wrote:
> On Mon, 22 Jan 2024 14:37:32 +0200 (EET)
> Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> 
> > On Mon, 22 Jan 2024, Jonathan Woithe wrote:
> > 
> > > On Sun, Jan 21, 2024 at 02:54:22PM +0200, Andy Shevchenko wrote:  
> > > > On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:  
> > > > > On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:  
> > > > > > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:  
> > > > > > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:  
> > > > > > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > > > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > > > >   
> > > > > > > > > Hi all,
> > > > > > > > > 
> > > > > > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > > > > > disparity in BAR sizes.
> > > > > > > > > 
> > > > > > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > > > > > decision (currently that function is called find_resource()). In order
> > > > > > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > > > > > few extra patches on resource side.
> > > > > > > > > 
> > > > > > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > > > > > the issues related to the currently ongoing resource regression
> > > > > > > > > thread [1].  
> > > > > > > > 
> > > > > > > > Jonathan,
> > > > > > > > can you test this series on affected machine with broken kernel to see if
> > > > > > > > it's of any help in your case?  
> > > > > > > 
> > > > > > > Certainly, but it will have to wait until next Thursday (11 Jan 2024).  I'm
> > > > > > > still on leave this week, and when at work I only have physical access to
> > > > > > > the machine concerned on Thursdays at present.
> > > > > > > 
> > > > > > > Which kernel would you prefer I apply the series to?  
> > > > > > 
> > > > > > I was very short of time today but I did apply the above series to the
> > > > > > 5.15.y branch (since I had this source available), resulting in version
> > > > > > 5.15.141+.  Unfortunately, in the rush I forgot to do a clean after the
> > > > > > bisect reset, so the resulting kernel was not correctly built.  It booted
> > > > > > but thought it was a different version and therefore none of the modules
> > > > > > could be found.  As a result, the test is invalid.
> > > > > > 
> > > > > > I will try again in a week when I next have physical access to the system. 
> > > > > > Apologies for the delay.  In the meantime, if there's a specific kernel I
> > > > > > should apply the patch series against please let me know.  As I understand
> > > > > > it, you want it applied to one of the kernels which failed, making 5.15.y
> > > > > > (for y < 145) a reasonable choice.  
> > > > > 
> > > > > I did a "make clean" to reset the source tree and recompiled.  However, it
> > > > > errored out:
> > > > > 
> > > > >   drivers/pci/setup-bus.c:988:24: error: ‘RESOURCE_SIZE_MAX’ undeclared
> > > > >   drivers/pci/setup-bus.c:998:17: error: ‘pci_bus_for_each_resource’ undeclared
> > > > > 
> > > > > This was with the patch series applied against 5.15.141.  It seems the patch
> > > > > targets a kernel that's too far removed from 5.15.x.
> > > > > 
> > > > > Which kernel would you like me to apply the patch series to and test?  
> > > > 
> > > > The rule of thumb is to test against latest vanilla (as of today v6.7).
> > > > Also makes sense to test against Linux Next. The v5.15 is way too old for
> > > > a new code.  
> > > 
> > > Thanks, and understood.  In this case the request from Igor was 
> > > 
> > >     can you test this series on affected machine with broken kernel to see if
> > >     it's of any help in your case?
> > > 
> > > The latest vanilla kernel (6.7) has (AFAIK) had the offending commit
> > > reverted, so it's not a "broken" kernel in this respect.  Therefore, if I've
> > > understood the request correctly, working with that kernel won't produce the
> > > desired test.  
> > 
> > Well, you can revert the revert again to get back to the broken state.
> 
> either this or just a hand patching as Ilpo has suggested earlier
> would do.

No problem.  This was the easiest approach for me and I have now done this. 
Apologies for the delay in getting to this: I ran out of time last Thursday.

> There is non zero chance that this series might fix issues
> Jonathan is facing. i.e. failed resource reallocation which
> offending patches trigger.

I can confirm that as expected, this patch series has had no effect on the
system which experiences the failed resource reallocation.  From syslog,
running a 5.15.141+ kernel[1]:

    kernel: radeon 0000:4b:00.0: Fatal error during GPU init
    kernel: radeon: probe of 0000:4b:00.0 failed with error -12

This is unchanged from what is seen with the unaltered 5.15.141 kernel.

In case it's important, can also confirm that the errors related to the
thunderbolt device are are also still present in the patched 5.15.141+
kernel:

    thunderbolt 0000:04:00.0: interrupt for TX ring 0 is already enabled
    :
    thunderbolt 0000:04:00.0: interrupt for RX ring 0 is already enabled
    :

Like the GPU failure, they do not appear in the working kernels on this
system.

Let me know if you would like to me to run further tests.

Regards
  jonathan

[1] This is 5.15.141, patched with the series of interest here and the hand
    patch from Ilpo.
  

On Thu, 1 Feb 2024, Jonathan Woithe wrote:

> On Mon, Jan 22, 2024 at 02:45:20PM +0100, Igor Mammedov wrote:
> > On Mon, 22 Jan 2024 14:37:32 +0200 (EET)
> > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > 
> > > On Mon, 22 Jan 2024, Jonathan Woithe wrote:
> > > 
> > > > On Sun, Jan 21, 2024 at 02:54:22PM +0200, Andy Shevchenko wrote:  
> > > > > On Thu, Jan 18, 2024 at 05:18:45PM +1030, Jonathan Woithe wrote:  
> > > > > > On Thu, Jan 11, 2024 at 06:30:22PM +1030, Jonathan Woithe wrote:  
> > > > > > > On Thu, Jan 04, 2024 at 10:48:53PM +1030, Jonathan Woithe wrote:  
> > > > > > > > On Thu, Jan 04, 2024 at 01:12:10PM +0100, Igor Mammedov wrote:  
> > > > > > > > > On Thu, 28 Dec 2023 18:57:00 +0200
> > > > > > > > > Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> wrote:
> > > > > > > > >   
> > > > > > > > > > Hi all,
> > > > > > > > > > 
> > > > > > > > > > Here's a series that contains two fixes to PCI bridge window sizing
> > > > > > > > > > algorithm. Together, they should enable remove & rescan cycle to work
> > > > > > > > > > for a PCI bus that has PCI devices with optional resources and/or
> > > > > > > > > > disparity in BAR sizes.
> > > > > > > > > > 
> > > > > > > > > > For the second fix, I chose to expose find_empty_resource_slot() from
> > > > > > > > > > kernel/resource.c because it should increase accuracy of the cannot-fit
> > > > > > > > > > decision (currently that function is called find_resource()). In order
> > > > > > > > > > to do that sensibly, a few improvements seemed in order to make its
> > > > > > > > > > interface and name of the function sane before exposing it. Thus, the
> > > > > > > > > > few extra patches on resource side.
> > > > > > > > > > 
> > > > > > > > > > Unfortunately I don't have a reason to suspect these would help with
> > > > > > > > > > the issues related to the currently ongoing resource regression
> > > > > > > > > > thread [1].  

> > > > Thanks, and understood.  In this case the request from Igor was 
> > > > 
> > > >     can you test this series on affected machine with broken kernel to see if
> > > >     it's of any help in your case?
> > > > 
> > > > The latest vanilla kernel (6.7) has (AFAIK) had the offending commit
> > > > reverted, so it's not a "broken" kernel in this respect.  Therefore, if I've
> > > > understood the request correctly, working with that kernel won't produce the
> > > > desired test.  
> > > 
> > > Well, you can revert the revert again to get back to the broken state.
> > 
> > either this or just a hand patching as Ilpo has suggested earlier
> > would do.
> 
> No problem.  This was the easiest approach for me and I have now done this. 
> Apologies for the delay in getting to this: I ran out of time last Thursday.
> 
> > There is non zero chance that this series might fix issues
> > Jonathan is facing. i.e. failed resource reallocation which
> > offending patches trigger.
> 
> I can confirm that as expected, this patch series has had no effect on the
> system which experiences the failed resource reallocation.  From syslog,
> running a 5.15.141+ kernel[1]:
> 
>     kernel: radeon 0000:4b:00.0: Fatal error during GPU init
>     kernel: radeon: probe of 0000:4b:00.0 failed with error -12
> 
> This is unchanged from what is seen with the unaltered 5.15.141 kernel.
> 
> In case it's important, can also confirm that the errors related to the
> thunderbolt device are are also still present in the patched 5.15.141+
> kernel:
> 
>     thunderbolt 0000:04:00.0: interrupt for TX ring 0 is already enabled
>     :
>     thunderbolt 0000:04:00.0: interrupt for RX ring 0 is already enabled
>     :
> 
> Like the GPU failure, they do not appear in the working kernels on this
> system.
> 
> Let me know if you would like to me to run further tests.
> 
> Regards
>   jonathan
> 
> [1] This is 5.15.141, patched with the series of interest here and the hand
>     patch from Ilpo.

Hi Jonathan,

Thanks a lot for testing it regardless. The end result was not a big 
surprise given how it looked like based on the logs but was certainly 
worth a test like Igor mentioned. The resource allocation code isn't among 
the easiest to track.