Набор данных / JSON, извлекаемый внутри цикла Foreach, очень медленный, не похоже на проблему с вызовом базы данных

Question

Это вызов базы данных -

 if (status == ValuationFilter.All)
                {
                    valuations = db.Valuations.Where(v => ((v.Creationdate >= startDate && v.Creationdate <= endDate) ||
                        v.SI47Instruction.Count(s => s.RequestDateTime >= startDate && s.RequestDateTime <= endDate) > 0)).Include(t => t.SI47Instruction).ToArray();
                }

Ниже приведен цикл Foreach, где данные собираются и отправляются на веб-интерфейс

            foreach (var item in valuations)
            {
                var valuationActiveAction = Action.GetActiveActionByValuationAndCompany(item.Id, companyId);
                var valuationReport = Report.GetByValuationId(item.Id);

                var dashboardItem = new Domain.DTO.DashboardItem
                {
                    //PropertyTitle = item.Property.Address1,
                    //PropertyFullAddress = String.Format("{0} {1} {2}", item.Property.Address1, item.Property.Address2, item.Property.City),
                    PropertyTitle = (String.Format("{0}{1} {2}{3} {4}", item.Property.Address1.Replace(",", ""), ",", String.IsNullOrEmpty(item.Property.Address2) ? "" : item.Property.Address2.Replace(",", ""), ",", item.Property.City)).Replace(", ,", ","),
                    PropertyFullAddress = (String.Format("{0}{1} {2}{3} {4}", item.Property.Address1.Replace(",", ""), ",", String.IsNullOrEmpty(item.Property.Address2) ? "" : item.Property.Address2.Replace(",", ""), ",", item.Property.City)).Replace(", ,", ","),
                    Originator = db.Companies.Where(c => c.Id == item.OriginationCompanyId).First().Name,
                    Valuer = item.ValuationCompanyId > 0 ? String.Format("{0} {1} {2}", db.Companies.Where(c => c.Id == item.ValuationCompanyId).First().Name, "-", db.Companies.Where(c => c.Id == item.ValuationCompanyId).First().Telephone) : "",
                    Notes = item.Note,
                    Action = valuationActiveAction.Type,
                    BorrowersName = item.BorrowerName
                };

                if (item.HasBeenDeclined.HasValue && item.HasBeenDeclined.Value && companyType == CompanyType.VMSI)
                {
                    var declinedReasons = ValuationDeclined.GetByValuationIdForDashboard(item.Id);
                    dashboardItem.DeclineReasons = declinedReasons.Length > 0 ? declinedReasons : null;
                }

                if (item.SI47Instruction.Count > 0)
                {
                    var lastestSI47 = item.SI47Instruction.OrderByDescending(s => s.Id).FirstOrDefault();

                    switch(status)
                    {
                        case ValuationFilter.Completed:
                            if (lastestSI47.Status != (int)InstructionStatus.SI47ReportComplete)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.CompletedWithDrawn:
                            if (lastestSI47.Status != (int)InstructionStatus.SI47ReportComplete &&
                                lastestSI47.Status != (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgress:
                            if (lastestSI47.Status == (int)InstructionStatus.SI47ReportComplete ||
                                lastestSI47.Status == (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgressCompleted:
                            if (lastestSI47.Status == (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgressWithdrawn:
                            if (lastestSI47.Status == (int)InstructionStatus.SI47ReportComplete)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.Withdrawn:
                            if (lastestSI47.Status != (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                    }

                    List<SI47.Instruction> si47Instructions = new List<SI47.Instruction>();
                    foreach (var si47Instruction in item.SI47Instruction)
                    {
                        si47Instructions.Add(new SI47.Instruction(si47Instruction));
                    }

                    if (si47Instructions.Count > 0)
                    {
                        dashboardItem.Type = InstructionType.SI47;

                        var activeSI47 = si47Instructions.OrderByDescending(c => c.Creationdate).First();
                        var activeSI47InstructionReport = SI47.Report.GetByInstructionId(activeSI47.Id);


                        dashboardItem.RequestId = activeSI47.Id;
                        dashboardItem.StandardInstructionId = activeSI47.StandardRequestId;
                        dashboardItem.Status = (Domain.Enumerations.InstructionEnumerations.InstructionStatus)activeSI47.Status;
                        dashboardItem.Date = activeSI47.Creationdate;

                        if (SuperBank >= 1)
                        {
                            dashboardItem.ReportId = activeSI47InstructionReport.Id;
                        }
                        else
                        {
                            dashboardItem.ReportId = companyType == Domain.Enumerations.Security.CompanyType.Origination ? 0 : activeSI47InstructionReport.Id;
                        }
                        dashboardItem.CompletionDate = activeSI47.DateReportSigned;

                        if (companyType == CompanyType.Origination || companyType == CompanyType.VMSI)
                        {
                            dashboardItem.RevalDate = activeSI47.RevalDate;
                        }

                        List<Domain.DTO.DashboardPreviousInstruction> prevInstructionList = new List<Domain.DTO.DashboardPreviousInstruction>();
                        // add Standard Request to list of previous instructions
                        prevInstructionList.Add(new Domain.DTO.DashboardPreviousInstruction()
                            {
                                Id = item.Id,
                                Alerts = Business.Valuation.Alert.GetDescriptionsForMultipleToDashboard(valuationReport.Alerts.Select(i => (int)i).ToArray()),
                                InspectionDate = item.DateInspection,
                                InstructionDate = item.Creationdate,
                                ReportCompletedDate = item.DateReportSigned.Value,
                                ReportId = valuationReport.Id,
                                AppointmentDate = item.Deadline,
                                Type = "Standard"
                            });

                        // add remaining SI47 Instruction to previous instructions
                        if (si47Instructions.Count > 1)
                        {
                            var previousSI47List = si47Instructions.Where(t => !t.IsActive).ToArray();

                            foreach (var prevSI47 in previousSI47List)
                            {
                                var prevSI47InstructionReport = SI47.Report.GetByInstructionId(prevSI47.Id);

                                prevInstructionList.Add(new Domain.DTO.DashboardPreviousInstruction()
                                {
                                    Id = prevSI47.Id,
                                    Alerts = new string[0],
                                    InstructionDate = prevSI47.Creationdate,
                                    ReportCompletedDate = prevSI47.DateReportSigned.Value,
                                    ReportId = prevSI47InstructionReport.Id,
                                    Type = "SI47"
                                });
                            }
                        }

                        dashboardItem.PreviousInstructions = prevInstructionList.ToArray();
                    }
                }
                else
                {
                    dashboardItem.Type = InstructionType.Standard;
                    dashboardItem.StandardInstructionId = item.Id;
                    dashboardItem.RequestId = item.Id;
                    dashboardItem.Status = (Domain.Enumerations.InstructionEnumerations.InstructionStatus)item.Status;
                    dashboardItem.Date = item.Creationdate;
                    dashboardItem.Appointment = item.Deadline;
                    if (SuperBank >= 1)
                    {
                        dashboardItem.ReportId = valuationReport.Id;
                    }
                    else
                    {
                        dashboardItem.ReportId = companyType == Domain.Enumerations.Security.CompanyType.Origination ? 0 : valuationReport.Id;
                    }
                    dashboardItem.CompletionDate = item.DateReportSigned;
                    dashboardItem.InspectionDate = item.DateInspection;

                    if (item.HasAlerts.HasValue && item.HasAlerts.Value)
                    {
                        dashboardItem.Alerts = Business.Valuation.Alert.GetDescriptionsForMultipleToDashboard(valuationReport.Alerts.Select(i => (int)i).ToArray());
                    }

                    if (companyType == CompanyType.Origination || companyType == CompanyType.VMSI)
                    {
                        dashboardItem.RevalDate = item.RevalDate;
                    }

                    dashboardItem.PreviousInstructions = new Domain.DTO.DashboardPreviousInstruction[0];

                }

                result.Add(dashboardItem);
            }
        }

        return result.OrderBy(i => i.Date).ToArray();
    }


            foreach (var item in valuations)
            {
                var valuationActiveAction = Action.GetActiveActionByValuationAndCompany(item.Id, companyId);
                var valuationReport = Report.GetByValuationId(item.Id);

                var dashboardItem = new Domain.DTO.DashboardItem
                {
                    //PropertyTitle = item.Property.Address1,
                    //PropertyFullAddress = String.Format("{0} {1} {2}", item.Property.Address1, item.Property.Address2, item.Property.City),
                    PropertyTitle = (String.Format("{0}{1} {2}{3} {4}", item.Property.Address1.Replace(",", ""), ",", String.IsNullOrEmpty(item.Property.Address2) ? "" : item.Property.Address2.Replace(",", ""), ",", item.Property.City)).Replace(", ,", ","),
                    PropertyFullAddress = (String.Format("{0}{1} {2}{3} {4}", item.Property.Address1.Replace(",", ""), ",", String.IsNullOrEmpty(item.Property.Address2) ? "" : item.Property.Address2.Replace(",", ""), ",", item.Property.City)).Replace(", ,", ","),
                    Originator = db.Companies.Where(c => c.Id == item.OriginationCompanyId).First().Name,
                    Valuer = item.ValuationCompanyId > 0 ? String.Format("{0} {1} {2}", db.Companies.Where(c => c.Id == item.ValuationCompanyId).First().Name, "-", db.Companies.Where(c => c.Id == item.ValuationCompanyId).First().Telephone) : "",
                    Notes = item.Note,
                    Action = valuationActiveAction.Type,
                    BorrowersName = item.BorrowerName
                };

                if (item.HasBeenDeclined.HasValue && item.HasBeenDeclined.Value && companyType == CompanyType.VMSI)
                {
                    var declinedReasons = ValuationDeclined.GetByValuationIdForDashboard(item.Id);
                    dashboardItem.DeclineReasons = declinedReasons.Length > 0 ? declinedReasons : null;
                }

                if (item.SI47Instruction.Count > 0)
                {
                    var lastestSI47 = item.SI47Instruction.OrderByDescending(s => s.Id).FirstOrDefault();

                    switch(status)
                    {
                        case ValuationFilter.Completed:
                            if (lastestSI47.Status != (int)InstructionStatus.SI47ReportComplete)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.CompletedWithDrawn:
                            if (lastestSI47.Status != (int)InstructionStatus.SI47ReportComplete &&
                                lastestSI47.Status != (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgress:
                            if (lastestSI47.Status == (int)InstructionStatus.SI47ReportComplete ||
                                lastestSI47.Status == (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgressCompleted:
                            if (lastestSI47.Status == (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.InProgressWithdrawn:
                            if (lastestSI47.Status == (int)InstructionStatus.SI47ReportComplete)
                            {
                                continue;
                            }
                            break;
                        case ValuationFilter.Withdrawn:
                            if (lastestSI47.Status != (int)InstructionStatus.Withdrawn)
                            {
                                continue;
                            }
                            break;
                    }

                    List<SI47.Instruction> si47Instructions = new List<SI47.Instruction>();
                    foreach (var si47Instruction in item.SI47Instruction)
                    {
                        si47Instructions.Add(new SI47.Instruction(si47Instruction));
                    }

                    if (si47Instructions.Count > 0)
                    {
                        dashboardItem.Type = InstructionType.SI47;

                        var activeSI47 = si47Instructions.OrderByDescending(c => c.Creationdate).First();
                        var activeSI47InstructionReport = SI47.Report.GetByInstructionId(activeSI47.Id);


                        dashboardItem.RequestId = activeSI47.Id;
                        dashboardItem.StandardInstructionId = activeSI47.StandardRequestId;
                        dashboardItem.Status = (Domain.Enumerations.InstructionEnumerations.InstructionStatus)activeSI47.Status;
                        dashboardItem.Date = activeSI47.Creationdate;

                        if (SuperBank >= 1)
                        {
                            dashboardItem.ReportId = activeSI47InstructionReport.Id;
                        }
                        else
                        {
                            dashboardItem.ReportId = companyType == Domain.Enumerations.Security.CompanyType.Origination ? 0 : activeSI47InstructionReport.Id;
                        }
                        dashboardItem.CompletionDate = activeSI47.DateReportSigned;

                        if (companyType == CompanyType.Origination || companyType == CompanyType.VMSI)
                        {
                            dashboardItem.RevalDate = activeSI47.RevalDate;
                        }

                        List<Domain.DTO.DashboardPreviousInstruction> prevInstructionList = new List<Domain.DTO.DashboardPreviousInstruction>();
                        // add Standard Request to list of previous instructions
                        prevInstructionList.Add(new Domain.DTO.DashboardPreviousInstruction()
                            {
                                Id = item.Id,
                                Alerts = Business.Valuation.Alert.GetDescriptionsForMultipleToDashboard(valuationReport.Alerts.Select(i => (int)i).ToArray()),
                                InspectionDate = item.DateInspection,
                                InstructionDate = item.Creationdate,
                                ReportCompletedDate = item.DateReportSigned.Value,
                                ReportId = valuationReport.Id,
                                AppointmentDate = item.Deadline,
                                Type = "Standard"
                            });

                        // add remaining SI47 Instruction to previous instructions
                        if (si47Instructions.Count > 1)
                        {
                            var previousSI47List = si47Instructions.Where(t => !t.IsActive).ToArray();

                            foreach (var prevSI47 in previousSI47List)
                            {
                                var prevSI47InstructionReport = SI47.Report.GetByInstructionId(prevSI47.Id);

                                prevInstructionList.Add(new Domain.DTO.DashboardPreviousInstruction()
                                {
                                    Id = prevSI47.Id,
                                    Alerts = new string[0],
                                    InstructionDate = prevSI47.Creationdate,
                                    ReportCompletedDate = prevSI47.DateReportSigned.Value,
                                    ReportId = prevSI47InstructionReport.Id,
                                    Type = "SI47"
                                });
                            }
                        }

                        dashboardItem.PreviousInstructions = prevInstructionList.ToArray();
                    }
                }
                else
                {
                    dashboardItem.Type = InstructionType.Standard;
                    dashboardItem.StandardInstructionId = item.Id;
                    dashboardItem.RequestId = item.Id;
                    dashboardItem.Status = (Domain.Enumerations.InstructionEnumerations.InstructionStatus)item.Status;
                    dashboardItem.Date = item.Creationdate;
                    dashboardItem.Appointment = item.Deadline;
                    if (SuperBank >= 1)
                    {
                        dashboardItem.ReportId = valuationReport.Id;
                    }
                    else
                    {
                        dashboardItem.ReportId = companyType == Domain.Enumerations.Security.CompanyType.Origination ? 0 : valuationReport.Id;
                    }
                    dashboardItem.CompletionDate = item.DateReportSigned;
                    dashboardItem.InspectionDate = item.DateInspection;

                    if (item.HasAlerts.HasValue && item.HasAlerts.Value)
                    {
                        dashboardItem.Alerts = Business.Valuation.Alert.GetDescriptionsForMultipleToDashboard(valuationReport.Alerts.Select(i => (int)i).ToArray());
                    }

                    if (companyType == CompanyType.Origination || companyType == CompanyType.VMSI)
                    {
                        dashboardItem.RevalDate = item.RevalDate;
                    }

                    dashboardItem.PreviousInstructions = new Domain.DTO.DashboardPreviousInstruction[0];

                }

                result.Add(dashboardItem);
            }
        }

        return result.OrderBy(i => i.Date).ToArray();
    }

ПРИМЕЧАНИЕ: я запускал этот фрагмент кода много раз, первоначальное извлечение базы данных занимает всего 0,3 секунды, независимо от того, сколько элементов я извлекаю из базы данных, но создание списка внутри foreach занимает очень много времени, для некоторых Запросы до 2 минут, не в состоянии выяснить, что является причиной этой проблемы.

Answer 1

Является ли db вызовом базы данных? возможно EF? Если это так, то вы запрашиваете базу данных столько раз, сколько Length() из valuations.

Вы делаете это в строках , у вас есть следующий код:

Originator = db.Companies.Where(c => c.Id == tem.OriginationCompanyId).First().Name

Вот почему это невероятно медленно.

У меня нет доступа к вашему коду, поэтому ниже приведено очень простое решение, которое может / не может применяться в вашем случае использования, но имейте в виду, что ваша проблема находится в строке выше . Вы должны избегать любой ценой вызова базы данных внутри цикла.

Вы должны быть в состоянии решить эту проблему, включив ссылку на OriginationCompany типа Companies внутри Valuations класса сущности (я предполагаю, что вы используете EntityFramework, и он имеет FK CompanyID в оценках как обычный сценарий будет).

Затем, когда вы впервые запрашиваете свою базу данных, вам нужно просто добавить «Включить для компании» следующим образом:

valuations = db.Valuations.Where(v => ((v.Creationdate >= startDate && v.Creationdate <= endDate) || 
    v.SI47Instruction.Count(s => s.RequestDateTime >= startDate 
    && s.RequestDateTime <= endDate) > 0))
    .Include(t => t.SI47Instruction)
    .Include(c => c.OriginationCompany)
    .ToArray();

Он будет загружен с нетерпением, и вы сможете сделать это при создании нового DashboardItem

Originator = item.OriginatorCompany.Name,