So we have 15million data is this table :allocation_plan_detail. I have put all necessary column in the non-clustered index and rest all other inside Include statement is while querying we are selecting *.
Here is the Index defination :
CREATE NONCLUSTERED INDEX [newNonClusteredIndex] ON [Allocation].[allocation_plan_detail]
(
[tenant_id] ASC,
[item_type] ASC,
[allocation_plan_status] ASC,
[allocation_plan_type] ASC,
[item_nbr] ASC,
[club_nbr] ASC
)
INCLUDE([item_config_id],[club_name],[dept_nbr],[subcat_nbr],[dc_nbr],[total_dc_on_hand_qty],[vc_nbr],[city_addr],[state_addr],[item_status_code],[base_unit_rtl_amt],[item_on_shelf_date],[item_off_shelf_date],[on_hand_qty],[on_order_qty],[total_inventory],[wkly_sales],[total_sales],[days_with_inventory],[avg_wos_planner],[forecast_level],[club_decile_nbr],[decile_rank],[current_wos],[projected_wos],[wos_target],[min_pres_qty],[max_inv_level],[oos_threshold],[moq],[lead_time],[mabd],[unconstrained_need_qty],[true_need_qty],[adjusted_planner_qty],[manual_override_planner_qty],[allocation_seq_nbr],[created_by],[created_on],[last_updated_by],[last_updated_on])
Now I am querying this :
select * from Allocation.allocation_plan_detail --with(index(newNonClusteredIndex))
where
allocation_plan_status='draft' and
item_type ='In season' and tenant_id='sams_us' and
allocation_plan_type='continuous' and
item_nbr in (980109489,326156,678518,980098926,980143632,299324,916205,207317,338064,433702,980427979,....upto 200 elements..)
With explicit index mention the plan:
Now even though I properly mentioned indexes why it scanning table? Now by some research in SO I am assuming its being done because IN clause has too many elements. But even with Index Seeking my query is taking on average to give output of 50k data for 5-6min avg!! Now I am completely clueless what exactly to do optimize the query ? I have tried with storing the IN elements into temporary table but yet I dont see any improvement. Any suggestions ?
Here is the table defination (Model)
public class AllocationPlanDetail extends BaseEntity{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#NotNull(groups = Existing.class)
#Null(groups = New.class)
private Long allocationPlanDetailId;
#NotNull
private Long itemConfigId;
#NotNull
private Long itemNbr;
#NotNull
private Integer clubNbr;
private String clubName;
private Integer deptNbr;
private Integer subcatNbr;
private Integer dcNbr;
private Integer totalDcOnHandQty;
private Integer vcNbr;
private String cityAddr;
private String stateAddr;
private String itemStatusCode;
private Float baseUnitRtlAmt;
private Date itemOnShelfDate;
private Date itemOffShelfDate;
private Integer onHandQty;
private Integer onOrderQty;
private Integer totalInventory;
#Column(name="wkly_sales",columnDefinition="nvarchar")
#Convert(converter = SalesConverterJson.class)
private Map<String,Float> wklySales;
private Float totalSales;
private Integer daysWithInventory;
#Column(name="avg_wos_planner")
private Float avgWOSPlanner;
private String forecastLevel;
private Integer clubDecileNbr;
private Integer decileRank;
#Column(name="current_wos")
private Float currentWOS;
#Column(name="projected_wos")
private Float projectedWOS;
private Integer wosTarget;
#Column(name="min_pres_qty")
private Integer minPres;
#Column(name="max_inv_level")
private Integer maxClubQty;
private Integer oosThreshold;
private Integer moq;
private Integer leadTime;
private Date mabd;
private Integer unconstrainedNeedQty;
private Integer trueNeedQty;
private Integer adjustedPlannerQty;
private Integer manualOverridePlannerQty;
private Integer allocationSeqNbr;
private String allocationPlanType;
private String allocationPlanStatus;
private String itemType;
#NotNull
private String tenantId;
I have asked similar quetion here : Getting Index Scan instead Index Seeking if IN clause larger in Azure Sql
But that time I havent included the INCLUDE clause in my index FYI. And second image got duplicated apology for that..
Related
enter image description here
Currently I've been problem what get my database data.. Because That's fileIdx Column was be Comma Seperated value
Problem is..
Caused by: java.sql.SQLException: Out of range value for column 'fileidx7_4_' : value 322,323
at org.mariadb.jdbc.internal.com.read.resultset.rowprotocol.TextRowProtocol.getInternalLong(TextRowProtocol.java:349)
at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.getLong(SelectResultSet.java:1001)
at org.mariadb.jdbc.internal.com.read.resultset.SelectResultSet.getLong(SelectResultSet.java:995)
at com.zaxxer.hikari.pool.HikariProxyResultSet.getLong(HikariProxyResultSet.java)
at org.hibernate.type.descriptor.sql.BigIntTypeDescriptor$2.doExtract(BigIntTypeDescriptor.java:63)
at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:47)
at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:257)
at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:253)
at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:243)
at org.hibernate.type.AbstractStandardBasicType.hydrate(AbstractStandardBasicType.java:329)
at org.hibernate.type.ManyToOneType.hydrate(ManyToOneType.java:184)
at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:3088)
at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1907)
at org.hibernate.loader.Loader.hydrateEntityState(Loader.java:1835)
at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1808)
at org.hibernate.loader.Loader.getRow(Loader.java:1660)
at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:745)
at org.hibernate.loader.Loader.getRowsFromResultSet(Loader.java:1044)
at org.hibernate.loader.Loader.processResultSet(Loader.java:995)
at org.hibernate.loader.Loader.doQuery(Loader.java:964)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:350)
at org.hibernate.loader.Loader.doList(Loader.java:2887)
... 108 more
It is my Entity ↓↓↓↓↓
#Entity
public class Board {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long idx;
private Long code;
private Long adminIdx;
private String write;
private String title;
private String content;
private String fileIdx;
private String deleted;
#Column(name = "sDate")
private Long sdate;
#Column(name = "eDate")
private Long edate;
private Long regdate;
It is my JpaRepository Method used JPQL ↓↓↓↓↓
#Query(value = "select b from Board b")
List<Board> getTest();
It is my TestCode ↓↓↓↓↓
#Test
public void test(){
List<Board> shopBoardEntity = boardRepository.getTest();
shopBoardEntity.forEach(o -> {
System.out.println("dddd : " + o.getTitle());
});
}
No matter how hard I try, I don't know..
Anyone Idea??
Can someone help me how should I join those three tables using JPA?
I already did 2 of 3 entities but please let me know if are ok:
#Entity
public class Pacienti {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private int id;
private String nume;
private String prenume;
//setters & getters
}
#Entity
public class Chestionare {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private int id;
#Id
#Column(name = "id_intrebare")
#GeneratedValue(strategy = GenerationType.AUTO)
private int idIntrebare;
private String intrebare;
//setters & getters
}
As I promise I come back after I'm generating entities automatically. Unfortunately now I have another problem.
Now I have the entity:
#Entity
#Table(name = "pacienti")
#NamedQuery(name = "Pacienti.findAll", query = "SELECT p FROM Pacienti p")
public class Pacienti implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(unique = true, nullable = false)
private int id;
#Column(nullable = false, length = 20)
private String nume;
#Column(nullable = false, length = 20)
private String prenume;
// bi-directional many-to-one association to Consultatii
#OneToMany(mappedBy = "pacienti")
private List<Consultatii> consultatiis;
// bi-directional many-to-one association to DetaliiPacient
#OneToMany(mappedBy = "pacienti")
private List<DetaliiPacient> detaliiPacients;
// bi-directional many-to-one association to Doctori
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "id_doctor", nullable = false)
private Doctori doctori;
// bi-directional many-to-one association to RaspunsChestionar
#OneToMany(mappedBy = "pacienti")
private List<RaspunsChestionar> raspunsChestionars;
public Pacienti() {
}
//setters and getters
}
But when I do :
Query queryResult = sessionFactory.getCurrentSession().createQuery("from Pacienti");
I'm getting:
Pacienti is not mapped [from Pacienti] Error.
Can someone tell me why? I also tried "pacienti is not mapped [from pacienti]" but same result
Thank you!
I would recommend you to use the jpa tools/plugins available with the IDEs which will auto generate these jpa entities for you using the database tables rather than manually creating these.
And they will take care of setting the relationship b/w different entities(db tables) in the auto generation process itself.
If you are Eclipse you can achieve this.
The problem is bcz there is no query with the name "from pacienti" in place of that pass the query name "Pacienti.findAll" in your createQuery method.
Plz let ne know once you try this, if you face any problem
I have the following Entity :
#Entity
#Inheritance(strategy = InheritanceType.SINGLE_TABLE)
#DiscriminatorColumn(name = "DCOL", discriminatorType = DiscriminatorType.STRING)
#DiscriminatorValue("Alias")
public class Alias
{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key key;
#Basic
private UUID from;
#Basic
private UUID to;
#Extension(vendorName="datanucleus", key="gae.unindexed", value="true")
private Date createdOn;
public Alias() { /* intentionally blank */ }
public Alias(#Nonnull final UUID from, #Nonnull final UUID to)
{
this.key = KeyFactory.createKey(Alias.class.getSimpleName(), from.toString());
this.from = from;
this.to = to;
this.createdOn = new Date();
}
}
And here is the code that persists it :
final EntityManager em = EMF.TRANSACTIONS_OPTIONAL.createEntityManager();
try
{
final Alias a = new Alias(from, to);
em.persist(a);
}
finally
{
em.close();
}
Currently it takes 6 Write Operations to the Datastore to persist this Entity.
I reduced the number from 10 to 6 by marking createdOn with the #Extension to exclude it from the auto indexing. That is a 40% decrease in write ops!
Is there any way I can reduce the number of writes for something this simple?
Would using the low-level Datastore API directly make any improvement?
As you mentioned, your entity requires 6 write operations divided as follows:
1 write for the entity itself
1 write for the built-in EntitiesByKind index
2 writes for the from property (1 for the built-in index EntitiesByProperty and another for the built-in index EntitiesByPropertyDesc).
2 writes for the to property (1 for the built-in index EntitiesByProperty and another for the built-in index EntitiesByPropertyDesc).
The only opportunity you have at this point to reduce the number of writes is marking any of the from or to properties as unindexed as well (thus reducing the number of writes by 2 with each property).
You can see more information here: https://developers.google.com/appengine/docs/java/datastore/entities#Java_Understanding_write_costs
I have a Java model similar to:
public class Country {
#Id private String id;
private CurrencyId currencyId;
private List<Province> provinceList;
...
}
public class Province {
#Id private String id;
private Gobernor gobernorId;
private List<City> cityList;
...
}
public class City {
#Id private String id;
private String name;
...
}
I want to store that data using objectify. However, as Country data might change, I also want to store the date the Country data has been stored, so I think I should store an entity such as:
public class CountryListEntity {
#Id private String id;
private List<Country> countryList;
private Date storeDate;
}
Note I will only have one entity of kind CountryListEntity with the Id "root", if I can store it like that. I know very little about both how google apps stores data and how objectify works. I've tried many combinations of #Embedded, but I got many errors, i.e.
Cannot place array or collection properties inside #Embedded arrays or collections
Can anyone tell me how to define these classes? A snippet of the code needed to store and retrieve this "root" entity, would be highly appreciated!
#Embedded collections are transformed into a series of collection fields in the
low-level Entity. That's why one level embedding is all you can do.
If you are going to store/load all data at once and if your entities are as simple as the ones in your example you can put #Serialized annotation for your lists inside #Embedded lists.
You can find out more from this discussion.
The problem with this approach is that your low-level embeddings won't be able to be indexed.
public class CountryListEntity {
#Id private String id;
#Embedded
private List<Country> countryList;
private Date storeDate;
}
public class Country implements Serializable {
private String id;
private CurrencyId currencyId;
#Serialized
private List<Province> provinceList;
// ...
}
public class Province implements Serializable {
private String id;
private Gobernor gobernorId;
#Serialized
private List<City> cityList;
// ...
}
public class City implements Serializable {
private String id;
private String name;
// ...
}
is it possible to use Constraints in the google-app-engine? It seems not to work ...
http://www.datanucleus.org/products/accessplatform_1_1/jpa/orm/constr...
The properties codingSystem and code should be unique. Is there a
workaround?
#Entity
#Table(uniqueConstraints = {
#UniqueConstraint(columnNames = { "codingSystem", "code" }) })
public class ArticleCode {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key id;
private String codingSystem;
private String code;
Thanks,
Ralph
In a nutshell, no, they're not. The underlying datastore implementation doesn't support global transactions, so it's not practical to enforce arbitrary uniqueness constraints.
The workaround is to make the unique components part of the key name.
Thanks a lot, it works fine.
Here is my new code.
#Entity
public class ArticleCode {
#Id
private Key id;
#Column(name="codingSystem")
private String codingSystem;
#Column(name="code")
private String code;
public ArticleCode(Key parent, String codingSystem, String code) {
this.id = KeyFactory.createKey(parent, ArticleCode.class.getSimpleName(), codingSystem + code);
this.codingSystem = codingSystem;
this.code = code;
}